-----============= acceptance-small: sanity-pcc ============----- Thu May  1 18:05:33 EDT 2025
mgs: CentOS Linux release 7.9.2009 (Core)
MGS_OS_VERSION_ID=7
MGS_OS_ID=centos
MGS_OS_VERSION_CODE=117440512
MGS_OS_ID_LIKE=rhel fedora centos
mds1: CentOS Linux release 7.9.2009 (Core)
MDS1_OS_ID_LIKE=rhel fedora centos
MDS1_OS_ID=centos
MDS1_OS_VERSION_ID=7
MDS1_OS_VERSION_CODE=117440512
ost1: CentOS Linux release 7.9.2009 (Core)
OST1_OS_VERSION_CODE=117440512
OST1_OS_VERSION_ID=7
OST1_OS_ID_LIKE=rhel fedora centos
OST1_OS_ID=centos
client: CentOS Linux release 7.9.2009 (Core)
CLIENT_OS_ID=centos
CLIENT_OS_ID_LIKE=rhel fedora centos
CLIENT_OS_VERSION_ID=7
CLIENT_OS_VERSION_CODE=117440512
=== sanity-pcc: start setup 18:05:44 (1746137144) ===
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
oleg101-client.virtnet: executing check_config_client /mnt/lustre
oleg101-client.virtnet: Checking config lustre mounted on /mnt/lustre
Checking servers environments
Checking clients oleg101-client.virtnet environments
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800aa7da800.idle_timeout=debug
osc.lustre-OST0000-osc-ffff8800b5875800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800aa7da800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800b5875800.idle_timeout=debug
disable quota as required
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
=== sanity-pcc: finish setup 18:06:03 (1746137163) ===
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [true]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [touch] [/mnt/lustre/d0_runas_test/f7040]
nobody:x:99:
oleg101-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanity-pcc.*ex: No such file or directory
excepting tests: 
Killing existing copytools on oleg101-client.virtnet
pkill: pidfile not valid
Try `pkill --help' for more information.
Set HSM on and start
Set sanity-hsm HSM policy

== sanity-pcc test 1a: Test manual lfs pcc attach with manual HSM restore ========================================================== 18:06:29 (1746137189)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.213908 s, 245 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d1a.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: ff975ec5-69f3-44e3-8208-8de98fd786a9
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f1a.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=ff975ec5-69f3-44e3-8208-8de98fd786a9 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d1a.sanity-pcc/d1a.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
using SAVE_PROJECT_SUPPORTED=0
1+0 records in
1+0 records out
7654321 bytes (7.7 MB) copied, 0.0820454 s, 93.3 MB/s
Restore testing...
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Attach and detach testing
Start to detach the /mnt/lustre/d1a.sanity-pcc/f1a.sanity-pcc
Waiting 200s for 'SUCCEED'
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d1a.sanity-pcc is idle now
PASS 1a (17s)

== sanity-pcc test 1b: Test manual lfs pcc attach with restore on remote access ========================================================== 18:06:46 (1746137206)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128282 s, 409 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d1b.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 6682bbde-1641-4b84-ace3-7a1e5dba7c53
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f1b.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=6682bbde-1641-4b84-ace3-7a1e5dba7c53 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d1b.sanity-pcc/d1b.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
using SAVE_PROJECT_SUPPORTED=0
1+0 records in
1+0 records out
7654321 bytes (7.7 MB) copied, 0.055862 s, 137 MB/s
Restore testing...
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Attach and detach testing
Start to detach the /mnt/lustre/d1b.sanity-pcc/f1b.sanity-pcc
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d1b.sanity-pcc is idle now
PASS 1b (17s)

== sanity-pcc test 1c: Test automated attach using Project ID with manual HSM restore ========================================================== 18:07:03 (1746137223)
using SAVE_PROJECT_SUPPORTED=0
Stopping clients: oleg101-client.virtnet /mnt/lustre (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre opts:
Stopping clients: oleg101-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre2 opts:
Stopping /mnt/lustre-mds1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg101-server
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
Checking servers environments
Checking clients oleg101-client.virtnet environments
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
loading modules on: 'oleg101-server'
oleg101-server: oleg101-server.virtnet: executing load_modules_local
oleg101-server: Loading modules from /home/green/git/lustre-release/lustre
oleg101-server: detected 4 online CPUs by sysfs
oleg101-server: MODOPTS_LIBCFS=
oleg101-server: Force libcfs to create 2 CPU partitions
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory
Setup mgs, mdt, osts
Starting mds1: -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-MDT0000
Starting ost1: -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0000
Starting ost2: -o localrecov  lustre-ost2/ost2 /mnt/lustre-ost2
seq.cli-lustre-OST0001-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0001
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800aa7ca000.idle_timeout=debug
osc.lustre-OST0000-osc-ffff8800b60c3800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800aa7ca000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800b60c3800.idle_timeout=debug
disable quota as required
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.211772 s, 248 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d1c.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 831e2bd5-2c32-48b9-865f-63d8e1e66b6c
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f1c.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=831e2bd5-2c32-48b9-865f-63d8e1e66b6c (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d1c.sanity-pcc/d1c.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
1+0 records in
1+0 records out
7654321 bytes (7.7 MB) copied, 0.0793753 s, 96.4 MB/s
Restore testing...
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Attach and detach testing
Start to detach the /mnt/lustre/d1c.sanity-pcc/f1c.sanity-pcc
Waiting 200s for 'SUCCEED'
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d1c.sanity-pcc is idle now
PASS 1c (93s)

== sanity-pcc test 1d: Test Project ID with remote access ========================================================== 18:08:36 (1746137316)
using SAVE_PROJECT_SUPPORTED=0
Stopping clients: oleg101-client.virtnet /mnt/lustre (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre opts:
Stopping clients: oleg101-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre2 opts:
Stopping /mnt/lustre-mds1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg101-server
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
Checking servers environments
Checking clients oleg101-client.virtnet environments
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory
loading modules on: 'oleg101-server'
oleg101-server: oleg101-server.virtnet: executing load_modules_local
oleg101-server: Loading modules from /home/green/git/lustre-release/lustre
oleg101-server: detected 4 online CPUs by sysfs
oleg101-server: MODOPTS_LIBCFS=
oleg101-server: Force libcfs to create 2 CPU partitions
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory
Setup mgs, mdt, osts
Starting mds1: -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-MDT0000
Starting ost1: -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0000
Starting ost2: -o localrecov  lustre-ost2/ost2 /mnt/lustre-ost2
seq.cli-lustre-OST0001-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0001
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff88012c0ce000.idle_timeout=debug
osc.lustre-OST0000-osc-ffff88012e8ec000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff88012c0ce000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff88012e8ec000.idle_timeout=debug
disable quota as required
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106289 s, 493 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d1d.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 68a48518-f46e-4d70-a5dc-604b9f9ad002
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f1d.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=68a48518-f46e-4d70-a5dc-604b9f9ad002 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d1d.sanity-pcc/d1d.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
1+0 records in
1+0 records out
7654321 bytes (7.7 MB) copied, 0.910285 s, 8.4 MB/s
 sanity-pcc test_1d: @@@@@@ FAIL: expected /mnt/pcc.d1d.sanity-pcc/d1d.sanity-pcc/13a3/0x2000013a1:0x2:0x0 size: 7654321 got: 0 
  Trace dump:
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7249:error()
  = /home/green/git/lustre-release/lustre/tests/sanity-pcc.sh:92:check_file_size()
  = /home/green/git/lustre-release/lustre/tests/sanity-pcc.sh:102:check_lpcc_sizes()
  = /home/green/git/lustre-release/lustre/tests/sanity-pcc.sh:357:lpcc_rw_test()
  = /home/green/git/lustre-release/lustre/tests/sanity-pcc.sh:431:test_1d()
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7622:run_one()
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7685:run_one_logged()
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7503:run_test()
  = /home/green/git/lustre-release/lustre/tests/sanity-pcc.sh:433:main()
Dumping lctl log to /tmp/testlogs//sanity-pcc.test_1d.*.1746137381.log
Cleanup PCC backend on /mnt/lustre
rsync: chown "/tmp/testlogs/.sanity-pcc.test_1d.debug_log.oleg101-server.1746137381.log.l3dxAm" failed: Operation not permitted (1)
rsync: chown "/tmp/testlogs/.sanity-pcc.test_1d.dmesg.oleg101-server.1746137381.log.MmZYFq" failed: Operation not permitted (1)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1651) [generator=3.1.2]
/mnt/pcc.d1d.sanity-pcc is idle now
FAIL 1d (69s)

== sanity-pcc test 1e: Test RW-PCC with non-root user ==== 18:09:48 (1746137388)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-pcc.test_1e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
pcc:
  -
    pccpath: /tmp/arc1/sanity-pcc.test_1e/
    hsmtool: posix_v2
    rwid: 2
    roid: 0
    flags: 90
    autocache: projid={100}
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d1e.sanity-pcc/f1e.sanity-pcc] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.0111871 s, 91.5 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-w] [-i] [2] [/mnt/lustre/d1e.sanity-pcc/f1e.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d1e.sanity-pcc/f1e.sanity-pcc] [of=/dev/null] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00501092 s, 204 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/tests/truncate] [/mnt/lustre/d1e.sanity-pcc/f1e.sanity-pcc] [256]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/tests/truncate] [/mnt/lustre/d1e.sanity-pcc/f1e.sanity-pcc] [2048]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d1e.sanity-pcc/f1e.sanity-pcc] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00256565 s, 399 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [-k] [/mnt/lustre/d1e.sanity-pcc/f1e.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [touch] [/tmp/arc1/sanity-pcc.test_1e//13a4/0x2000013a1:0x5:0x0]
touch: cannot touch '/tmp/arc1/sanity-pcc.test_1e//13a4/0x2000013a1:0x5:0x0': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/tmp/arc1/sanity-pcc.test_1e//13a4/0x2000013a1:0x5:0x0] [of=/dev/null] [bs=1024] [count=1]
dd: failed to open '/tmp/arc1/sanity-pcc.test_1e//13a4/0x2000013a1:0x5:0x0': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/tmp/arc1/sanity-pcc.test_1e//13a4/0x2000013a1:0x5:0x0] [bs=1024] [count=1]
dd: failed to open '/tmp/arc1/sanity-pcc.test_1e//13a4/0x2000013a1:0x5:0x0': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-w] [-i] [2] [/mnt/lustre/d1e.sanity-pcc/f1e.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/d1e.sanity-pcc/f1e.sanity-pcc]
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Cleanup PCC backend on /mnt/lustre
PASS 1e (9s)

== sanity-pcc test 1f: Test auto RW-PCC cache with non-root user ========================================================== 18:09:57 (1746137397)
using SAVE_PROJECT_SUPPORTED=0
Stopping clients: oleg101-client.virtnet /mnt/lustre (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre opts:
Stopping clients: oleg101-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre2 opts:
Stopping /mnt/lustre-mds1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg101-server
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
Checking servers environments
Checking clients oleg101-client.virtnet environments
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory
loading modules on: 'oleg101-server'
oleg101-server: oleg101-server.virtnet: executing load_modules_local
oleg101-server: Loading modules from /home/green/git/lustre-release/lustre
oleg101-server: detected 4 online CPUs by sysfs
oleg101-server: MODOPTS_LIBCFS=
oleg101-server: Force libcfs to create 2 CPU partitions
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory
Setup mgs, mdt, osts
Starting mds1: -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-MDT0000
Starting ost1: -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0000
Starting ost2: -o localrecov  lustre-ost2/ost2 /mnt/lustre-ost2
seq.cli-lustre-OST0001-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0001
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800a99f6800.idle_timeout=debug
osc.lustre-OST0000-osc-ffff8800b6370000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800a99f6800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800b6370000.idle_timeout=debug
disable quota as required
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102734 s, 510 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d1f.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: a0445b46-ca25-4d21-a32a-0fb5f29ebba0
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f1f.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=a0445b46-ca25-4d21-a32a-0fb5f29ebba0 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d1f.sanity-pcc/d1f.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d1f.sanity-pcc/f1f.sanity-pcc] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00936265 s, 109 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d1f.sanity-pcc/f1f.sanity-pcc] [of=/dev/null] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00419907 s, 244 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/tests/truncate] [/mnt/lustre/d1f.sanity-pcc/f1f.sanity-pcc] [256]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/tests/truncate] [/mnt/lustre/d1f.sanity-pcc/f1f.sanity-pcc] [2048]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d1f.sanity-pcc/f1f.sanity-pcc] [bs=256] [count=1]
1+0 records in
1+0 records out
256 bytes (256 B) copied, 0.00457434 s, 56.0 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [touch] [/mnt/pcc.d1f.sanity-pcc/d1f.sanity-pcc/1b73/0x200001b71:0x2:0x0]
touch: cannot touch '/mnt/pcc.d1f.sanity-pcc/d1f.sanity-pcc/1b73/0x200001b71:0x2:0x0': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/pcc.d1f.sanity-pcc/d1f.sanity-pcc/1b73/0x200001b71:0x2:0x0] [of=/dev/null] [bs=1024] [count=1]
dd: failed to open '/mnt/pcc.d1f.sanity-pcc/d1f.sanity-pcc/1b73/0x200001b71:0x2:0x0': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/pcc.d1f.sanity-pcc/d1f.sanity-pcc/1b73/0x200001b71:0x2:0x0] [bs=1024] [count=1]
dd: failed to open '/mnt/pcc.d1f.sanity-pcc/d1f.sanity-pcc/1b73/0x200001b71:0x2:0x0': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/d1f.sanity-pcc/f1f.sanity-pcc]
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d1f.sanity-pcc is idle now
PASS 1f (82s)

== sanity-pcc test 1g: General permission test for RW-PCC ========================================================== 18:11:19 (1746137479)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108742 s, 482 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d1g.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: b66e0bae-4490-4004-9767-45acd03d3431
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f1g.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=b66e0bae-4490-4004-9767-45acd03d3431 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d1g.sanity-pcc/d1g.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.010588 s, 96.7 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/f1g.sanity-pcc] [bs=1024] [count=1]
dd: failed to open '/mnt/lustre/f1g.sanity-pcc': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/f1g.sanity-pcc] [of=/dev/null] [bs=1024] [count=1]
dd: failed to open '/mnt/lustre/f1g.sanity-pcc': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/f1g.sanity-pcc] [bs=1024] [count=1]
dd: failed to open '/mnt/lustre/f1g.sanity-pcc': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/f1g.sanity-pcc] [of=/dev/null] [bs=1024] [count=1]
dd: failed to open '/mnt/lustre/f1g.sanity-pcc': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/f1g.sanity-pcc] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00685255 s, 149 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/f1g.sanity-pcc]
detach: cannot detach '/mnt/lustre/f1g.sanity-pcc' from PCC: Operation not permitted
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/f1g.sanity-pcc]
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/f1g.sanity-pcc] [of=/dev/null] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.0145046 s, 70.6 kB/s
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d1g.sanity-pcc is idle now
PASS 1g (9s)

== sanity-pcc test 2a: Test multi open when creating ===== 18:11:28 (1746137488)
using SAVE_PROJECT_SUPPORTED=0
Stopping clients: oleg101-client.virtnet /mnt/lustre (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre opts:
Stopping clients: oleg101-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre2 opts:
Stopping /mnt/lustre-mds1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg101-server
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
Checking servers environments
Checking clients oleg101-client.virtnet environments
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
loading modules on: 'oleg101-server'
oleg101-server: oleg101-server.virtnet: executing load_modules_local
oleg101-server: Loading modules from /home/green/git/lustre-release/lustre
oleg101-server: detected 4 online CPUs by sysfs
oleg101-server: MODOPTS_LIBCFS=
oleg101-server: Force libcfs to create 2 CPU partitions
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory
Setup mgs, mdt, osts
Starting mds1: -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-MDT0000
Starting ost1: -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0000
Starting ost2: -o localrecov  lustre-ost2/ost2 /mnt/lustre-ost2
seq.cli-lustre-OST0001-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0001
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800aa7db000.idle_timeout=debug
osc.lustre-OST0000-osc-ffff8800b6c7d000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800aa7db000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800b6c7d000.idle_timeout=debug
disable quota as required
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117785 s, 445 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d2a.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 61bde830-2d8b-43d4-a0ed-b21df1b4b1a3
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f2a.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=61bde830-2d8b-43d4-a0ed-b21df1b4b1a3 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d2a.sanity-pcc/d2a.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
multiop /mnt/lustre/d2a.sanity-pcc/multiop vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.24654
client oleg101-client.virtnet multiop_bg started multiop_pid=24664
---------- 1 root root 14 May  1 18:12 /mnt/pcc.d2a.sanity-pcc/d2a.sanity-pcc/2343/0x200002341:0x2:0x0
Stopping multiop_pid=24664 (kill 24664 on oleg101-client.virtnet)
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d2a.sanity-pcc is idle now
PASS 2a (80s)

== sanity-pcc test 2b: Test multi remote open when creating ========================================================== 18:12:48 (1746137568)

 SKIP: sanity-pcc test_2b Need 2 or more clients, have 1
SKIP 2b (2s)

== sanity-pcc test 2c: Test multi open on different mount points when creating ========================================================== 18:12:50 (1746137570)
using SAVE_PROJECT_SUPPORTED=0
Stopping clients: oleg101-client.virtnet /mnt/lustre (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre opts:
Stopping clients: oleg101-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre2 opts:
Stopping /mnt/lustre-mds1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg101-server
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
Checking servers environments
Checking clients oleg101-client.virtnet environments
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory
loading modules on: 'oleg101-server'
oleg101-server: oleg101-server.virtnet: executing load_modules_local
oleg101-server: Loading modules from /home/green/git/lustre-release/lustre
oleg101-server: detected 4 online CPUs by sysfs
oleg101-server: MODOPTS_LIBCFS=
oleg101-server: Force libcfs to create 2 CPU partitions
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory
Setup mgs, mdt, osts
Starting mds1: -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-MDT0000
Starting ost1: -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0000
Starting ost2: -o localrecov  lustre-ost2/ost2 /mnt/lustre-ost2
seq.cli-lustre-OST0001-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0001
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800aa7d8800.idle_timeout=debug
osc.lustre-OST0000-osc-ffff88013690b000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800aa7d8800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff88013690b000.idle_timeout=debug
disable quota as required
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0964116 s, 544 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d2c.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 369e6fb4-085c-4c40-b95d-f610075d1800
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f2c.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=369e6fb4-085c-4c40-b95d-f610075d1800 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d2c.sanity-pcc/d2c.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
multiop /mnt/lustre/d2c.sanity-pcc/f2c.sanity-pcc vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.27844
client oleg101-client.virtnet multiop_bg started multiop_pid=27854
Stopping multiop_pid=27854 (kill 27854 on oleg101-client.virtnet)
multiopen_dataCleanup PCC backend on /mnt/lustre
/mnt/pcc.d2c.sanity-pcc is idle now
PASS 2c (66s)

== sanity-pcc test 3a: Repeat attach/detach operations === 18:13:56 (1746137636)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-pcc.test_3a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.0085965 s, 119 kB/s
Start to RW-PCC attach/detach the file: /mnt/lustre/d3a.sanity-pcc/f3a.sanity-pcc
Repeat to RW-PCC attach/detach the same file: /mnt/lustre/d3a.sanity-pcc/f3a.sanity-pcc
Start to RO-PCC attach/detach the file: /mnt/lustre/d3a.sanity-pcc/f3a.sanity-pcc
Repeat to RO-PCC attach/detach the same file: /mnt/lustre/d3a.sanity-pcc/f3a.sanity-pcc
Cleanup PCC backend on /mnt/lustre
PASS 3a (5s)

== sanity-pcc test 3b: Repeat attach/detach operations on multiple clients ========================================================== 18:14:01 (1746137641)

 SKIP: sanity-pcc test_3b Need 3 or more clients, have 1
SKIP 3b (2s)

== sanity-pcc test 4: Auto cache test for mmap =========== 18:14:03 (1746137643)
using SAVE_PROJECT_SUPPORTED=0
Stopping clients: oleg101-client.virtnet /mnt/lustre (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre opts:
Stopping clients: oleg101-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre2 opts:
Stopping /mnt/lustre-mds1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg101-server
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
Checking servers environments
Checking clients oleg101-client.virtnet environments
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory
loading modules on: 'oleg101-server'
oleg101-server: oleg101-server.virtnet: executing load_modules_local
oleg101-server: Loading modules from /home/green/git/lustre-release/lustre
oleg101-server: detected 4 online CPUs by sysfs
oleg101-server: MODOPTS_LIBCFS=
oleg101-server: Force libcfs to create 2 CPU partitions
oleg101-server: libkmod: kmod_module_get_size: failed to read coresize from /sys/module/pcc_cpufreq
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory
Setup mgs, mdt, osts
Starting mds1: -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-MDT0000
Starting ost1: -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0000
Starting ost2: -o localrecov  lustre-ost2/ost2 /mnt/lustre-ost2
seq.cli-lustre-OST0001-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0001
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800aa7c9800.idle_timeout=debug
osc.lustre-OST0000-osc-ffff880136ae2000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800aa7c9800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff880136ae2000.idle_timeout=debug
disable quota as required
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145636 s, 360 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d4.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 3e05f12c-d015-491b-adaa-baa813b89c59
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f4.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=3e05f12c-d015-491b-adaa-baa813b89c59 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d4.sanity-pcc/d4.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
mmap test1: basic mmap operation (PASS, 0.029359s)
mmap test2: MAP_PRIVATE not write back (PASS, 0.037127s)
mmap test3: concurrent mmap ops on two nodes (PASS, 2.0402s)
mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.1318s)
mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.024957s)
mmap test6: check mmap write/read content on two nodes (PASS, 1.002s)
mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s)
mmap test8: SIGBUS for beyond file size (SKIPPED, 0s)
mmap test9: SIGBUS for negative file offset (SKIPPED, 0s)
mmap test10: multi-thread mmap access (PASS, 0.15133s)
mmap test11: multi-thread shared mmap (PASS, 0.19798s)
mmap test10: mtime not change for readonly mmap access (PASS, 1.0338s)
ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa7c9800.lru_size=clear
ldlm.namespaces.lustre-MDT0000-mdc-ffff880136ae2000.lru_size=clear
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d4.sanity-pcc is idle now
PASS 4 (84s)

== sanity-pcc test 5: Mmap & cat a RW-PCC cached file ==== 18:15:27 (1746137727)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0863099 s, 607 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d5.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 14849992-10e8-4cd6-bb32-c2c18bda05b2
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f5.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=14849992-10e8-4cd6-bb32-c2c18bda05b2 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d5.sanity-pcc/d5.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
Waiting 200s for 'SUCCEED'
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d5.sanity-pcc is idle now
PASS 5 (7s)

== sanity-pcc test 6: Test mmap write on RW-PCC ========== 18:15:35 (1746137735)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.177313 s, 296 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d6.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: a4189e52-392f-4cb1-a622-4e5f9b8b3714
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f6.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=a4189e52-392f-4cb1-a622-4e5f9b8b3714 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d6.sanity-pcc/d6.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d6.sanity-pcc is idle now
PASS 6 (9s)

== sanity-pcc test 7a: Fake file detached between fault() and page_mkwrite() for RW-PCC ========================================================== 18:15:44 (1746137744)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145956 s, 359 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d7a.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: ecd7a6a8-2a53-47a7-bb91-3589bd92d752
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f7a.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=ecd7a6a8-2a53-47a7-bb91-3589bd92d752 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d7a.sanity-pcc/d7a.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
fail_loc=0x1412
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d7a.sanity-pcc is idle now
PASS 7a (8s)

== sanity-pcc test 7b: Test the race with concurrent mkwrite and detach ========================================================== 18:15:52 (1746137752)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128334 s, 409 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d7b.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: f8957aef-4771-4adf-be4a-1b429cb62407
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f7b.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=f8957aef-4771-4adf-be4a-1b429cb62407 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d7b.sanity-pcc/d7b.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
fail_loc=0x1413
fail_val=20
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d7b.sanity-pcc is idle now
PASS 7b (47s)

== sanity-pcc test 8: Test fake -ENOSPC tolerance for RW-PCC ========================================================== 18:16:40 (1746137800)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0984533 s, 533 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d8.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 96c54804-2301-45e4-a10a-75be61de1d46
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f8.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=96c54804-2301-45e4-a10a-75be61de1d46 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d8.sanity-pcc/d8.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
fail_loc=0x1411
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d8.sanity-pcc is idle now
PASS 8 (7s)

== sanity-pcc test 9: Test -ENOSPC tolerance on loop PCC device for RW-PCC ========================================================== 18:16:46 (1746137806)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0937068 s, 559 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.9a: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 5b7fc473-5bd4-451b-a4b4-bf8eb091d66b
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f9.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=5b7fc473-5bd4-451b-a4b4-bf8eb091d66b (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.9a/d9.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.9a/d9.sanity-pcc
    hsmtool: posix_v2
    rwid: 2
    roid: 0
    flags: 9e
    autocache: projid={100}
60+0 records in
60+0 records out
62914560 bytes (63 MB) copied, 2.15466 s, 29.2 MB/s
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.9a is idle now
PASS 9 (9s)

== sanity-pcc test 10a: Test RW-PCC with user quota on loop PCC device ========================================================== 18:16:55 (1746137815)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142754 s, 367 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d10a.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: b12f744d-10f6-4893-8359-3b33cc163cea
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f10a.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=b12f744d-10f6-4893-8359-3b33cc163cea (extents) (64bit) (large files) (huge files)
*** Report for user quotas on device /dev/loop0
Block grace time: 7days; Inode grace time: 7days
                        Space limits                File limits
User            used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --     13K      0K      0K              2     0     0       
sanityusr --      0K      0K  20480K              0     0     0       

Statistics:
Total blocks: 7
Data blocks: 1
Entries: 2
Used average: 2.000000

Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d10a.sanity-pcc/d10a.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.d10a.sanity-pcc/d10a.sanity-pcc
    hsmtool: posix_v2
    rwid: 2
    roid: 0
    flags: 9e
    autocache: projid={100}
15+0 records in
15+0 records out
15728640 bytes (16 MB) copied, 0.233011 s, 67.5 MB/s
15+0 records in
15+0 records out
15728640 bytes (16 MB) copied, 0.265734 s, 59.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-i] [2] [-w] [/mnt/lustre/d10a.sanity-pcc/uquotaA]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-i] [2] [-w] [/mnt/lustre/d10a.sanity-pcc/uquotaB]
lfs pcc pcc: cannot get UNLOCK lease, ext 10 for [0x2000032e1:0x26:0x0]: Disk quota exceeded (122)
lfs pcc pcc: cannot attach with ID: 2: Disk quota exceeded (122)
attach: cannot attach '/mnt/lustre/d10a.sanity-pcc/uquotaB' to PCC with attach ID '2': Disk quota exceeded
Test -EDQUOT error tolerance for RW-PCC
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d10a.sanity-pcc/uquotaA] [bs=1M] [count=30]
30+0 records in
30+0 records out
31457280 bytes (31 MB) copied, 1.69601 s, 18.5 MB/s
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d10a.sanity-pcc is idle now
PASS 10a (9s)

== sanity-pcc test 10b: Test RW-PCC with group quota on loop PCC device ========================================================== 18:17:04 (1746137824)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112753 s, 465 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d10b.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: b1af2607-b792-4a5a-8e76-34223fc62ef1
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f10b.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=b1af2607-b792-4a5a-8e76-34223fc62ef1 (extents) (64bit) (large files) (huge files)
*** Report for group quotas on device /dev/loop0
Block grace time: 7days; Inode grace time: 7days
                        Space limits                File limits
Group           used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --     13K      0K      0K              2     0     0       
sanityusr --      0K      0K  20480K              0     0     0       

Statistics:
Total blocks: 7
Data blocks: 1
Entries: 2
Used average: 2.000000

Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d10b.sanity-pcc/d10b.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.d10b.sanity-pcc/d10b.sanity-pcc
    hsmtool: posix_v2
    rwid: 2
    roid: 0
    flags: 9e
    autocache: projid={100}
15+0 records in
15+0 records out
15728640 bytes (16 MB) copied, 0.225794 s, 69.7 MB/s
15+0 records in
15+0 records out
15728640 bytes (16 MB) copied, 0.211529 s, 74.4 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-i] [2] [-w] [/mnt/lustre/d10b.sanity-pcc/gquotaA]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-i] [2] [-w] [/mnt/lustre/d10b.sanity-pcc/gquotaB]
lfs pcc pcc: cannot get UNLOCK lease, ext 10 for [0x2000032e1:0x2b:0x0]: Disk quota exceeded (122)
lfs pcc pcc: cannot attach with ID: 2: Disk quota exceeded (122)
attach: cannot attach '/mnt/lustre/d10b.sanity-pcc/gquotaB' to PCC with attach ID '2': Disk quota exceeded
Test -EDQUOT error tolerance for RW-PCC
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d10b.sanity-pcc/gquotaA] [bs=1M] [count=30]
30+0 records in
30+0 records out
31457280 bytes (31 MB) copied, 1.316 s, 23.9 MB/s
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d10b.sanity-pcc is idle now
PASS 10b (8s)

== sanity-pcc test 10c: Test RO-PCC with user quota on loop PCC device ========================================================== 18:17:13 (1746137833)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107375 s, 488 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d10c.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: fe4edd5a-5eba-43fd-8cc8-fb4e216926be
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f10c.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=fe4edd5a-5eba-43fd-8cc8-fb4e216926be (extents) (64bit) (large files) (huge files)
*** Report for user quotas on device /dev/loop0
Block grace time: 7days; Inode grace time: 7days
                        Space limits                File limits
User            used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --     13K      0K      0K              2     0     0       
sanityusr --      0K      0K  20480K              0     0     0       

Statistics:
Total blocks: 7
Data blocks: 1
Entries: 2
Used average: 2.000000

Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d10c.sanity-pcc/d10c.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.d10c.sanity-pcc/d10c.sanity-pcc
    hsmtool: posix_v2
    rwid: 2
    roid: 2
    flags: ae
    autocache: projid={100}
15+0 records in
15+0 records out
15728640 bytes (16 MB) copied, 0.226837 s, 69.3 MB/s
15+0 records in
15+0 records out
15728640 bytes (16 MB) copied, 0.160927 s, 97.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-i] [2] [/mnt/lustre/d10c.sanity-pcc/uquotaA]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-i] [2] [/mnt/lustre/d10c.sanity-pcc/uquotaB]
lfs pcc pcc: cannot attach the file to PCC with ID 2 failed: Disk quota exceeded (122)
attach: cannot attach '/mnt/lustre/d10c.sanity-pcc/uquotaB' to PCC with attach ID '2': Disk quota exceeded
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d10c.sanity-pcc is idle now
PASS 10c (6s)

== sanity-pcc test 10d: Test RO-PCC with group quota on loop PCC device ========================================================== 18:17:19 (1746137839)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111381 s, 471 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d10d.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: a6675dea-89b3-4109-becd-c3e0c54f41b2
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f10d.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=a6675dea-89b3-4109-becd-c3e0c54f41b2 (extents) (64bit) (large files) (huge files)
*** Report for group quotas on device /dev/loop0
Block grace time: 7days; Inode grace time: 7days
                        Space limits                File limits
Group           used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --     13K      0K      0K              2     0     0       
sanityusr --      0K      0K  20480K              0     0     0       

Statistics:
Total blocks: 7
Data blocks: 1
Entries: 2
Used average: 2.000000

Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d10d.sanity-pcc/d10d.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.d10d.sanity-pcc/d10d.sanity-pcc
    hsmtool: posix_v2
    rwid: 2
    roid: 2
    flags: ae
    autocache: projid={100}
15+0 records in
15+0 records out
15728640 bytes (16 MB) copied, 0.211691 s, 74.3 MB/s
15+0 records in
15+0 records out
15728640 bytes (16 MB) copied, 0.237501 s, 66.2 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-i] [2] [/mnt/lustre/d10d.sanity-pcc/gquotaA]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-i] [2] [/mnt/lustre/d10d.sanity-pcc/gquotaB]
lfs pcc pcc: cannot attach the file to PCC with ID 2 failed: Disk quota exceeded (122)
attach: cannot attach '/mnt/lustre/d10d.sanity-pcc/gquotaB' to PCC with attach ID '2': Disk quota exceeded
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d10d.sanity-pcc is idle now
PASS 10d (8s)

== sanity-pcc test 10e: Tolerate -EDQUOT failure when auto PCC-RO attach with user quota ========================================================== 18:17:27 (1746137847)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.17854 s, 294 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d10e.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 32c94b5c-c494-4a74-93be-45c4af363b83
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f10e.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=32c94b5c-c494-4a74-93be-45c4af363b83 (extents) (64bit) (large files) (huge files)
*** Report for user quotas on device /dev/loop0
Block grace time: 7days; Inode grace time: 7days
                        Space limits                File limits
User            used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --     13K      0K      0K              2     0     0       
sanityusr --      0K      0K   4096K              0     0     0       

Statistics:
Total blocks: 7
Data blocks: 1
Entries: 2
Used average: 2.000000

pcc:
  -
    pccpath: /mnt/pcc.d10e.sanity-pcc/d10e.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: uid={500}
2+0 records in
2+0 records out
2097152 bytes (2.1 MB) copied, 0.0367782 s, 57.0 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/f10e.sanity-pcc] [of=/dev/null] [bs=1M] [count=2]
2+0 records in
2+0 records out
2097152 bytes (2.1 MB) copied, 0.0117538 s, 178 MB/s
/mnt/lustre/f10e.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    2
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x2000032e1
      lmm_object_id:     0x34
      lmm_fid:           [0x2000032e1:0x34:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0xcf:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
5+0 records in
5+0 records out
5242880 bytes (5.2 MB) copied, 0.0763172 s, 68.7 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/f10e.sanity-pcc] [of=/dev/null] [bs=1M] [count=5]
5+0 records in
5+0 records out
5242880 bytes (5.2 MB) copied, 0.0320417 s, 164 MB/s
file: /mnt/lustre/f10e.sanity-pcc, type: none
/mnt/lustre/f10e.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         wp,pccro
  lcm_layout_gen:    5
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x2000032e1
      lmm_object_id:     0x34
      lmm_fid:           [0x2000032e1:0x34:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0xcf:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d10e.sanity-pcc is idle now
PASS 10e (6s)

== sanity-pcc test 10f: Tolerate -EDQUOT failure when auto PCC-RO attach with group quota ========================================================== 18:17:33 (1746137853)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0958674 s, 547 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d10f.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 1cfdd3f8-cd7b-4e52-8a16-6d94923dcb2e
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f10f.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=1cfdd3f8-cd7b-4e52-8a16-6d94923dcb2e (extents) (64bit) (large files) (huge files)
*** Report for group quotas on device /dev/loop0
Block grace time: 7days; Inode grace time: 7days
                        Space limits                File limits
Group           used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --     13K      0K      0K              2     0     0       
sanityusr --      0K      0K   4096K              0     0     0       

Statistics:
Total blocks: 7
Data blocks: 1
Entries: 2
Used average: 2.000000

pcc:
  -
    pccpath: /mnt/pcc.d10f.sanity-pcc/d10f.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: gid={500}
2+0 records in
2+0 records out
2097152 bytes (2.1 MB) copied, 0.0526575 s, 39.8 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/f10f.sanity-pcc] [of=/dev/null] [bs=1M] [count=2]
2+0 records in
2+0 records out
2097152 bytes (2.1 MB) copied, 0.00775557 s, 270 MB/s
/mnt/lustre/f10f.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    2
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x2000032e1
      lmm_object_id:     0x36
      lmm_fid:           [0x2000032e1:0x36:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0xcf:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
5+0 records in
5+0 records out
5242880 bytes (5.2 MB) copied, 0.0972777 s, 53.9 MB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/f10f.sanity-pcc] [of=/dev/null] [bs=1M] [count=5]
5+0 records in
5+0 records out
5242880 bytes (5.2 MB) copied, 0.024111 s, 217 MB/s
file: /mnt/lustre/f10f.sanity-pcc, type: none
/mnt/lustre/f10f.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         wp,pccro
  lcm_layout_gen:    5
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x2000032e1
      lmm_object_id:     0x36
      lmm_fid:           [0x2000032e1:0x36:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0xcf:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d10f.sanity-pcc is idle now
PASS 10f (8s)

== sanity-pcc test 11: Test attach fault injection with simulated PCC file path ========================================================== 18:17:41 (1746137861)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.213149 s, 246 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d11.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 1eee4b29-4297-4ac5-a45b-4801d1d7714b
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f11.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=1eee4b29-4297-4ac5-a45b-4801d1d7714b (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d11.sanity-pcc/d11.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
Lustre file: /mnt/lustre/f11.sanity-pcc LPCC dir: /mnt/pcc.d11.sanity-pcc/d11.sanity-pcc/32d9
Lustre file: /mnt/lustre/f11.sanity-pcc LPCC dir: /mnt/pcc.d11.sanity-pcc/d11.sanity-pcc/32db
lfs pcc pcc: cannot get UNLOCK lease, ext 10 for [0x2000032e1:0x3a:0x0]: Permission denied (13)
lfs pcc pcc: cannot attach with ID: 2: Permission denied (13)
attach: cannot attach '/mnt/lustre/f11.sanity-pcc' to PCC with attach ID '2': Permission denied
lfs pcc pcc: cannot get UNLOCK lease, ext 10 for [0x2000032e1:0x3b:0x0]: Invalid argument (22)
lfs pcc pcc: cannot attach with ID: 2: Invalid argument (22)
attach: cannot attach '/mnt/lustre/f11.sanity-pcc' to PCC with attach ID '2': Invalid argument
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d11.sanity-pcc is idle now
PASS 11 (6s)

== sanity-pcc test 12: RW-PCC attach races with concurrent HSM remove ========================================================== 18:17:47 (1746137867)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-pcc.test_12/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
fail_loc=0x1414
fail_val=20
/mnt/lustre/f12.sanity-pcc: (0x0000000d) released exists archived, archive_id:2
Cleanup PCC backend on /mnt/lustre
PASS 12 (25s)

== sanity-pcc test 13a: Test auto RW-PCC create caching for UID/GID rule ========================================================== 18:18:12 (1746137892)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-pcc.test_13a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
pcc:
  -
    pccpath: /tmp/arc1/sanity-pcc.test_13a/
    hsmtool: posix_v2
    rwid: 2
    roid: 0
    flags: 90
    autocache: uid={500}
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d13a.sanity-pcc/f13a.sanity-pcc] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00692 s, 148 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d13a.sanity-pcc/f13a.sanity-pcc] [of=/dev/null] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00480596 s, 213 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/tests/truncate] [/mnt/lustre/d13a.sanity-pcc/f13a.sanity-pcc] [256]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/tests/truncate] [/mnt/lustre/d13a.sanity-pcc/f13a.sanity-pcc] [2048]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d13a.sanity-pcc/f13a.sanity-pcc] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00540617 s, 189 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/d13a.sanity-pcc/f13a.sanity-pcc]
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Cleanup PCC backend on /mnt/lustre
pcc:
  -
    pccpath: /tmp/arc1/sanity-pcc.test_13a/
    hsmtool: posix_v2
    rwid: 2
    roid: 0
    flags: 90
    autocache: gid={500}
lfs mkdir: dirstripe error on '/mnt/lustre/d13a.sanity-pcc': stripe already set
lfs setdirstripe: cannot create dir '/mnt/lustre/d13a.sanity-pcc': File exists
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d13a.sanity-pcc/f13a.sanity-pcc] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00464663 s, 220 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d13a.sanity-pcc/f13a.sanity-pcc] [of=/dev/null] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00519696 s, 197 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/tests/truncate] [/mnt/lustre/d13a.sanity-pcc/f13a.sanity-pcc] [256]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/tests/truncate] [/mnt/lustre/d13a.sanity-pcc/f13a.sanity-pcc] [2048]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d13a.sanity-pcc/f13a.sanity-pcc] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00338429 s, 303 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/d13a.sanity-pcc/f13a.sanity-pcc]
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Cleanup PCC backend on /mnt/lustre
Cleanup PCC backend on /mnt/lustre
Cleanup PCC backend on /mnt/lustre
PASS 13a (13s)

== sanity-pcc test 13b: Test auto RW-PCC create caching for file name with wildcard ========================================================== 18:18:25 (1746137905)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-pcc.test_13b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
pcc:
  -
    pccpath: /tmp/arc1/sanity-pcc.test_13b/
    hsmtool: posix_v2
    rwid: 2
    roid: 0
    flags: 90
    autocache: fname={*.h5 suffix.* Mid*dle}
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00715859 s, 143 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d13b.sanity-pcc/suffix.doc] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00740191 s, 138 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d13b.sanity-pcc/MidPADdle] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00581931 s, 176 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d13b.sanity-pcc/Midpad] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.0107958 s, 94.9 kB/s
Cleanup PCC backend on /mnt/lustre
PASS 13b (6s)

== sanity-pcc test 13c: Check auto RW-PCC create caching for UID/GID/ProjID/fname rule ========================================================== 18:18:32 (1746137912)
using SAVE_PROJECT_SUPPORTED=0
Stopping clients: oleg101-client.virtnet /mnt/lustre (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre opts:
Stopping clients: oleg101-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre2 opts:
Stopping /mnt/lustre-mds1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg101-server
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
Checking servers environments
Checking clients oleg101-client.virtnet environments
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory
loading modules on: 'oleg101-server'
oleg101-server: oleg101-server.virtnet: executing load_modules_local
oleg101-server: Loading modules from /home/green/git/lustre-release/lustre
oleg101-server: detected 4 online CPUs by sysfs
oleg101-server: MODOPTS_LIBCFS=
oleg101-server: Force libcfs to create 2 CPU partitions
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory
Setup mgs, mdt, osts
Starting mds1: -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-MDT0000
Starting ost1: -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0000
Starting ost2: -o localrecov  lustre-ost2/ost2 /mnt/lustre-ost2
seq.cli-lustre-OST0001-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0001
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800aa7df800.idle_timeout=debug
osc.lustre-OST0000-osc-ffff88012d0da000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800aa7df800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff88012d0da000.idle_timeout=debug
disable quota as required
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110394 s, 475 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d13c.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 45455773-b777-4f15-93a5-a71302d194ba
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f13c.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=45455773-b777-4f15-93a5-a71302d194ba (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d13c.sanity-pcc/d13c.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.d13c.sanity-pcc/d13c.sanity-pcc
    hsmtool: posix_v2
    rwid: 2
    roid: 0
    flags: 9e
    autocache: projid={100 200}&fname={*.h5},uid={500}&gid={500}
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.0093014 s, 110 kB/s
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00610132 s, 168 kB/s
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.0182694 s, 56.0 kB/s
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00589382 s, 174 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d13c.sanity-pcc/ugidcache] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00474418 s, 216 kB/s
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d13c.sanity-pcc is idle now
PASS 13c (81s)

== sanity-pcc test 14: Revocation of the layout lock should detach the file automatically ========================================================== 18:19:53 (1746137993)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-pcc.test_14/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa7df800.lru_size=clear
ldlm.namespaces.lustre-MDT0000-mdc-ffff88012d0da000.lru_size=clear
ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa7df800.lru_size=clear
ldlm.namespaces.lustre-MDT0000-mdc-ffff88012d0da000.lru_size=clear
Cleanup PCC backend on /mnt/lustre
PASS 14 (6s)

== sanity-pcc test 15: Test auto attach at open when file is still valid cached ========================================================== 18:19:59 (1746137999)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12917 s, 406 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d15.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 8803ec11-5e41-4112-91f6-02974bffa5bf
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f15.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=8803ec11-5e41-4112-91f6-02974bffa5bf (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d15.sanity-pcc/d15.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
Verify open attach for non-root user
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d15.sanity-pcc/f15.sanity-pcc] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00667745 s, 153 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-w] [-i] [2] [/mnt/lustre/d15.sanity-pcc/f15.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [state] [/mnt/lustre/d15.sanity-pcc/f15.sanity-pcc]
file: /mnt/lustre/d15.sanity-pcc/f15.sanity-pcc, type: readwrite, PCC_file: /d15.sanity-pcc/3aa0/0x200003ab1:0x11:0x0, open_count: 0, flags: 0
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [state] [/mnt/lustre/d15.sanity-pcc/f15.sanity-pcc]
ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa7df800.lru_size=clear
ldlm.namespaces.lustre-MDT0000-mdc-ffff88012d0da000.lru_size=clear
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [state] [/mnt/lustre/d15.sanity-pcc/f15.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [multiop] [/mnt/lustre/d15.sanity-pcc/f15.sanity-pcc] [oc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [state] [/mnt/lustre/d15.sanity-pcc/f15.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [-k] [/mnt/lustre/d15.sanity-pcc/f15.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [state] [/mnt/lustre/d15.sanity-pcc/f15.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [multiop] [/mnt/lustre/d15.sanity-pcc/f15.sanity-pcc] [oc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [state] [/mnt/lustre/d15.sanity-pcc/f15.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/d15.sanity-pcc/f15.sanity-pcc]
Verify auto attach at open for RW-PCC
ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa7df800.lru_size=clear
ldlm.namespaces.lustre-MDT0000-mdc-ffff88012d0da000.lru_size=clear
Verify auto attach at open for RO-PCC
ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa7df800.lru_size=clear
ldlm.namespaces.lustre-MDT0000-mdc-ffff88012d0da000.lru_size=clear
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d15.sanity-pcc is idle now
PASS 15 (12s)

== sanity-pcc test 16: Test detach with different options ========================================================== 18:20:11 (1746138011)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124018 s, 423 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d16.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 53130882-f3d0-4bd8-a17f-fa3d8e0269b9
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f16.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=53130882-f3d0-4bd8-a17f-fa3d8e0269b9 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d16.sanity-pcc/d16.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
Test detach for RW-PCC
Test for reusing valid PCC cache
Test for the default detach
Waiting 200s for 'SUCCEED'
Test detach for RO-PCC
detach_dataCleanup PCC backend on /mnt/lustre
/mnt/pcc.d16.sanity-pcc is idle now
PASS 16 (11s)

== sanity-pcc test 17: Test auto attach for layout refresh ========================================================== 18:20:22 (1746138022)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.158479 s, 331 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d17.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 25312b1e-1f8b-405a-9085-0d379587340b
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f17.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=25312b1e-1f8b-405a-9085-0d379587340b (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d17.sanity-pcc/d17.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.d17.sanity-pcc/d17.sanity-pcc
    hsmtool: posix_v2
    rwid: 2
    roid: 0
    flags: 94
    autocache: projid={100}
Verify auto attach during IO for truncate
Verify auto attach during IO for read/write
multiop /mnt/lustre/f17.sanity-pcc vO_r
TMPPIPE=/tmp/multiop_open_wait_pipe.19368
client oleg101-client.virtnet multiop_bg started multiop_pid=19379
ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa7df800.lru_size=clear
ldlm.namespaces.lustre-MDT0000-mdc-ffff88012d0da000.lru_size=clear
Stopping multiop_pid=19379 (kill 19379 on oleg101-client.virtnet)
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d17.sanity-pcc is idle now
PASS 17 (16s)

== sanity-pcc test 18: Verify size correctness after re-attach the file ========================================================== 18:20:38 (1746138038)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112357 s, 467 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d18.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 4afcc981-bd98-410a-9bf9-fb3b47216639
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f18.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=4afcc981-bd98-410a-9bf9-fb3b47216639 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d18.sanity-pcc/d18.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.d18.sanity-pcc/d18.sanity-pcc
    hsmtool: posix_v2
    rwid: 2
    roid: 0
    flags: 9e
    autocache: projid={100}
4+0 records in
4+0 records out
4194304 bytes (4.2 MB) copied, 0.13121 s, 32.0 MB/s
file: /mnt/lustre/f18.sanity-pcc, type: readwrite, PCC_file: /d18.sanity-pcc/3aac/0x200003ab1:0x1d:0x0, open_count: 0, flags: 0
file: /mnt/lustre/f18.sanity-pcc, type: none
/mnt/lustre/f18.sanity-pcc has size 4194304 OK
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00422641 s, 242 kB/s
/mnt/lustre2/f18.sanity-pcc has size 1049600 OK
file: /mnt/lustre/f18.sanity-pcc, type: none
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d18.sanity-pcc is idle now
PASS 18 (8s)

== sanity-pcc test 19: Verify the file re-attach works as expected ========================================================== 18:20:46 (1746138046)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.135013 s, 388 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d19.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 40310af5-294a-4eed-b5cf-f3eac497048c
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f19.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=40310af5-294a-4eed-b5cf-f3eac497048c (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d19.sanity-pcc/d19.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d19.sanity-pcc is idle now
PASS 19 (7s)

== sanity-pcc test 20: Auto attach works after the inode was once evicted from cache ========================================================== 18:20:53 (1746138053)
120+0 records in
120+0 records out
125829120 bytes (126 MB) copied, 0.287768 s, 437 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d20.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:   1024/122880             done                            
Creating filesystem with 122880 1k blocks and 30720 inodes
Filesystem UUID: ce9462d0-36f8-4022-a95f-67f8bdc4a913
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345, 73729

Allocating group tables:  0/15     done                            
Writing inode tables:  0/15     done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information:  0/15     done

/tmp/f20.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=ce9462d0-36f8-4022-a95f-67f8bdc4a913 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d20.sanity-pcc/d20.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d20.sanity-pcc is idle now
PASS 20 (7s)

== sanity-pcc test 21a: PCC-RO storing as a plain HSM mirror component for plain layout ========================================================== 18:21:00 (1746138060)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0930459 s, 563 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d21a.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 3b4ed1c9-fcbd-4001-a7be-c9cb0d5f96ac
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f21a.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=3b4ed1c9-fcbd-4001-a7be-c9cb0d5f96ac (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d21a.sanity-pcc/d21a.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
Plain layout info before PCC-RO attach '/mnt/lustre/f21a.sanity-pcc':
/mnt/lustre/f21a.sanity-pcc
lmm_magic:         0x0BD10BD0
lmm_seq:           0x200003ab1
lmm_object_id:     0x29
lmm_fid:           [0x200003ab1:0x29:0x0]
lmm_stripe_count:  1
lmm_stripe_size:   4194304
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 0
	obdidx		 objid		 objid		 group
	     0	           267	        0x10b	   0x240000400


FLR layout info after PCC-RO attach '/mnt/lustre/f21a.sanity-pcc':
/mnt/lustre/f21a.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    2
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x29
      lmm_fid:           [0x200003ab1:0x29:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x10b:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''

FLR layout info after PCC-RO detach '/mnt/lustre/f21a.sanity-pcc':
/mnt/lustre/f21a.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    2
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x29
      lmm_fid:           [0x200003ab1:0x29:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x10b:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''

FLR layout info after RO-PCC attach /mnt/lustre/f21a.sanity-pcc again:
/mnt/lustre/f21a.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    2
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x29
      lmm_fid:           [0x200003ab1:0x29:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x10b:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''

FLR layout info after RO-PCC detach '/mnt/lustre/f21a.sanity-pcc' again:
/mnt/lustre/f21a.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    2
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x29
      lmm_fid:           [0x200003ab1:0x29:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x10b:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d21a.sanity-pcc is idle now
PASS 21a (5s)

== sanity-pcc test 21b: PCC-RO stroing as a plain HSM mirror component for FLR layouts ========================================================== 18:21:05 (1746138065)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0888515 s, 590 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d21b.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 3d538ea4-66b5-4267-a3cc-eb7afd287e38
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f21b.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=3d538ea4-66b5-4267-a3cc-eb7afd287e38 (extents) (64bit) (large files) (huge files)
FLR layout before PCC-RO attach '/mnt/lustre/f21b.sanity-pcc':
/mnt/lustre/f21b.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          288
  lcm_flags:         ro
  lcm_layout_gen:    2
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2a
      lmm_fid:           [0x200003ab1:0x2a:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x10c:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x10c:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         208
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2a
      lmm_fid:           [0x200003ab1:0x2a:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x10d:0x0] }
      -   1: { l_ost_idx:   0, l_fid: [0x240000400:0x10d:0x0] }


FLR layout after PCC-RO attach '/mnt/lustre/f21b.sanity-pcc':
/mnt/lustre/f21b.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          408
  lcm_flags:         ro,pccro
  lcm_layout_gen:    3
  lcm_mirror_count:  3
  lcm_entry_count:   3
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         176
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2a
      lmm_fid:           [0x200003ab1:0x2a:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x10c:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x10c:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         256
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2a
      lmm_fid:           [0x200003ab1:0x2a:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x10d:0x0] }
      -   1: { l_ost_idx:   0, l_fid: [0x240000400:0x10d:0x0] }

  - lcme_id:             196611
    lcme_mirror_id:      3
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         336
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''

FLR layout info after PCC-RO detach '/mnt/lustre/f21b.sanity-pcc':
/mnt/lustre/f21b.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          408
  lcm_flags:         ro,pccro
  lcm_layout_gen:    3
  lcm_mirror_count:  3
  lcm_entry_count:   3
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         176
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2a
      lmm_fid:           [0x200003ab1:0x2a:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x10c:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x10c:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         256
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2a
      lmm_fid:           [0x200003ab1:0x2a:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x10d:0x0] }
      -   1: { l_ost_idx:   0, l_fid: [0x240000400:0x10d:0x0] }

  - lcme_id:             196611
    lcme_mirror_id:      3
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         336
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''

FLR layout after PCC-RO attach '/mnt/lustre/f21b.sanity-pcc' again:
/mnt/lustre/f21b.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          408
  lcm_flags:         ro,pccro
  lcm_layout_gen:    3
  lcm_mirror_count:  3
  lcm_entry_count:   3
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         176
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2a
      lmm_fid:           [0x200003ab1:0x2a:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x10c:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x10c:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         256
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2a
      lmm_fid:           [0x200003ab1:0x2a:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x10d:0x0] }
      -   1: { l_ost_idx:   0, l_fid: [0x240000400:0x10d:0x0] }

  - lcme_id:             196611
    lcme_mirror_id:      3
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         336
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''

FLR layout info after PCC-RO detach '/mnt/lustre/f21b.sanity-pcc':
/mnt/lustre/f21b.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          408
  lcm_flags:         ro,pccro
  lcm_layout_gen:    3
  lcm_mirror_count:  3
  lcm_entry_count:   3
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         176
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2a
      lmm_fid:           [0x200003ab1:0x2a:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x10c:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x10c:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         256
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2a
      lmm_fid:           [0x200003ab1:0x2a:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x10d:0x0] }
      -   1: { l_ost_idx:   0, l_fid: [0x240000400:0x10d:0x0] }

  - lcme_id:             196611
    lcme_mirror_id:      3
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         336
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d21b.sanity-pcc is idle now
PASS 21b (6s)

== sanity-pcc test 21c: Verify HSM release works storing PCC-RO as HSM mirror component ========================================================== 18:21:11 (1746138071)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122563 s, 428 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d21c.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: a31808bc-89e5-4e05-a0d1-06507b0dc3da
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f21c.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=a31808bc-89e5-4e05-a0d1-06507b0dc3da (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d21c.sanity-pcc/d21c.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
/mnt/lustre/f21c.sanity-pcc: (0x00000009) exists archived, archive_id:2
/mnt/lustre/f21c.sanity-pcc
  lcm_layout_gen:    3
  lcm_mirror_count:  2
  lcm_entry_count:   2
    lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       released
      lmm_layout_gen:    0
      lmm_stripe_offset: 0

    lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
/mnt/lustre/f21c.sanity-pcc: (0x0000000d) released exists archived, archive_id:2
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d21c.sanity-pcc is idle now
PASS 21c (7s)

== sanity-pcc test 21d: Write should invalidate PCC-RO caching ========================================================== 18:21:18 (1746138078)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121746 s, 431 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d21d.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: c14f2635-b68a-4f96-bdcc-0639fe266f6a
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f21d.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=c14f2635-b68a-4f96-bdcc-0639fe266f6a (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d21d.sanity-pcc/d21d.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
/mnt/lustre/f21d.sanity-pcc
lmm_stripe_count:  1
lmm_stripe_size:   4194304
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 1
	obdidx		 objid		 objid		 group
	     1	           270	        0x10e	   0x280000400

PCC-RO attach '/mnt/lustre/f21d.sanity-pcc':
/mnt/lustre/f21d.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    2
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2d
      lmm_fid:           [0x200003ab1:0x2d:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x10e:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
Write invalidated PCC-RO cache:
/mnt/lustre/f21d.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         wp
  lcm_layout_gen:    4
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2d
      lmm_fid:           [0x200003ab1:0x2d:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x10e:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init,stale
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,dirty,archived,0xb
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d21d.sanity-pcc is idle now
PASS 21d (6s)

== sanity-pcc test 21e: Truncate should invalidate PCC-RO caching ========================================================== 18:21:24 (1746138084)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.143205 s, 366 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d21e.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: b4aed563-9b8a-4350-b744-be509b292cab
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f21e.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=b4aed563-9b8a-4350-b744-be509b292cab (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d21e.sanity-pcc/d21e.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
/mnt/lustre/f21e.sanity-pcc
lmm_stripe_count:  1
lmm_stripe_size:   4194304
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 0
	obdidx		 objid		 objid		 group
	     0	           271	        0x10f	   0x240000400

PCC-RO attach '/mnt/lustre/f21e.sanity-pcc':
/mnt/lustre/f21e.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    2
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2f
      lmm_fid:           [0x200003ab1:0x2f:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x10f:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    8594128896
      lhm_archive_ver:   9663677440
      lhm_archive_uuid:  ''
Trucate invalidate PCC-RO file '/mnt/lustre/f21e.sanity-pcc':
/mnt/lustre/f21e.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         wp
  lcm_layout_gen:    4
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x2f
      lmm_fid:           [0x200003ab1:0x2f:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x10f:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init,stale
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,dirty,archived,0xb
      lhm_archive_id:    8594128896
      lhm_archive_ver:   9663677440
      lhm_archive_uuid:  ''
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d21e.sanity-pcc is idle now
PASS 21e (6s)

== sanity-pcc test 21f: mmap write should invalidate PCC-RO caching ========================================================== 18:21:30 (1746138090)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124106 s, 422 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d21f.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 9cf5fe54-0758-4287-8007-e50cfae3675f
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f21f.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=9cf5fe54-0758-4287-8007-e50cfae3675f (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d21f.sanity-pcc/d21f.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
/mnt/lustre/f21f.sanity-pcc
lmm_stripe_count:  1
lmm_stripe_size:   4194304
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 1
	obdidx		 objid		 objid		 group
	     1	           271	        0x10f	   0x280000400

PCC-RO attach '/mnt/lustre/f21f.sanity-pcc':
/mnt/lustre/f21f.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    2
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x30
      lmm_fid:           [0x200003ab1:0x30:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x10f:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    8594128896
      lhm_archive_ver:   9663677440
      lhm_archive_uuid:  ''
Mmap write invalidate PCC-RO caching:
/mnt/lustre/f21f.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         wp
  lcm_layout_gen:    4
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x30
      lmm_fid:           [0x200003ab1:0x30:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x10f:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init,stale
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,dirty,archived,0xb
      lhm_archive_id:    8594128896
      lhm_archive_ver:   9663677440
      lhm_archive_uuid:  ''
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d21f.sanity-pcc is idle now
PASS 21f (6s)

== sanity-pcc test 21g: PCC-RO for file under FLR write pending state ========================================================== 18:21:36 (1746138096)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.114581 s, 458 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d21g.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: f6496e99-ad2c-43ab-aaa1-a951db3c10f4
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f21g.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=f6496e99-ad2c-43ab-aaa1-a951db3c10f4 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d21g.sanity-pcc/d21g.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
FLR layout before PCC-RO attach '/mnt/lustre/f21g.sanity-pcc':
/mnt/lustre/f21g.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          288
  lcm_flags:         wp
  lcm_layout_gen:    3
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x32
      lmm_fid:           [0x200003ab1:0x32:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x110:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x110:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init,stale
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         208
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x32
      lmm_fid:           [0x200003ab1:0x32:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x111:0x0] }
      -   1: { l_ost_idx:   0, l_fid: [0x240000400:0x111:0x0] }

FLR layout after PCC-RO attach '/mnt/lustre/f21g.sanity-pcc':
/mnt/lustre/f21g.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          408
  lcm_flags:         wp,pccro
  lcm_layout_gen:    4
  lcm_mirror_count:  3
  lcm_entry_count:   3
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         176
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x32
      lmm_fid:           [0x200003ab1:0x32:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x110:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x110:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init,stale
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         256
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x32
      lmm_fid:           [0x200003ab1:0x32:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x111:0x0] }
      -   1: { l_ost_idx:   0, l_fid: [0x240000400:0x111:0x0] }

  - lcme_id:             196612
    lcme_mirror_id:      3
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         336
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
Layout after Write invalidate '/mnt/lustre/f21g.sanity-pcc':
/mnt/lustre/f21g.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          408
  lcm_flags:         wp
  lcm_layout_gen:    5
  lcm_mirror_count:  3
  lcm_entry_count:   3
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         176
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x32
      lmm_fid:           [0x200003ab1:0x32:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x110:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x110:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init,stale
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         256
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x32
      lmm_fid:           [0x200003ab1:0x32:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x111:0x0] }
      -   1: { l_ost_idx:   0, l_fid: [0x240000400:0x111:0x0] }

  - lcme_id:             196612
    lcme_mirror_id:      3
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         336
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,0x9
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d21g.sanity-pcc is idle now
PASS 21g (6s)

== sanity-pcc test 21h: Extend mirror once file was PCC-RO cached ========================================================== 18:21:42 (1746138102)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.152343 s, 344 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d21h.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 90d40f2a-f5a6-4530-ad39-70f469e6dcb6
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f21h.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=90d40f2a-f5a6-4530-ad39-70f469e6dcb6 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d21h.sanity-pcc/d21h.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
FLR layout before PCC-RO attach '/mnt/lustre/f21h.sanity-pcc':
/mnt/lustre/f21h.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          288
  lcm_flags:         ro
  lcm_layout_gen:    2
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x35
      lmm_fid:           [0x200003ab1:0x35:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x112:0x0] }
      -   1: { l_ost_idx:   0, l_fid: [0x240000400:0x112:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         208
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x35
      lmm_fid:           [0x200003ab1:0x35:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x113:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x113:0x0] }


FLR layout after PCC-RO attach '/mnt/lustre/f21h.sanity-pcc':
/mnt/lustre/f21h.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          408
  lcm_flags:         ro,pccro
  lcm_layout_gen:    3
  lcm_mirror_count:  3
  lcm_entry_count:   3
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         176
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x35
      lmm_fid:           [0x200003ab1:0x35:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x112:0x0] }
      -   1: { l_ost_idx:   0, l_fid: [0x240000400:0x112:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         256
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x35
      lmm_fid:           [0x200003ab1:0x35:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x113:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x113:0x0] }

  - lcme_id:             196611
    lcme_mirror_id:      3
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         336
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''

FLR layout after extend a mirror:
/mnt/lustre/f21h.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          536
  lcm_flags:         ro,pccro
  lcm_layout_gen:    4
  lcm_mirror_count:  4
  lcm_entry_count:   4
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         224
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x35
      lmm_fid:           [0x200003ab1:0x35:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x112:0x0] }
      -   1: { l_ost_idx:   0, l_fid: [0x240000400:0x112:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         304
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x35
      lmm_fid:           [0x200003ab1:0x35:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x113:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x113:0x0] }

  - lcme_id:             196611
    lcme_mirror_id:      3
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         384
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
  - lcme_id:             262145
    lcme_mirror_id:      4
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         456
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x35
      lmm_fid:           [0x200003ab1:0x35:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   8388608
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x114:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x114:0x0] }

file: /mnt/lustre/f21h.sanity-pcc, type: none

FLR layout after PCC-RO attach '/mnt/lustre/f21h.sanity-pcc' again:
/mnt/lustre/f21h.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          536
  lcm_flags:         ro,pccro
  lcm_layout_gen:    4
  lcm_mirror_count:  4
  lcm_entry_count:   4
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         224
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x35
      lmm_fid:           [0x200003ab1:0x35:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x112:0x0] }
      -   1: { l_ost_idx:   0, l_fid: [0x240000400:0x112:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         304
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x35
      lmm_fid:           [0x200003ab1:0x35:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x113:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x113:0x0] }

  - lcme_id:             196611
    lcme_mirror_id:      3
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         384
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
  - lcme_id:             262145
    lcme_mirror_id:      4
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         456
    lcme_size:           80
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x35
      lmm_fid:           [0x200003ab1:0x35:0x0]
      lmm_stripe_count:  2
      lmm_stripe_size:   8388608
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x114:0x0] }
      -   1: { l_ost_idx:   1, l_fid: [0x280000400:0x114:0x0] }

Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d21h.sanity-pcc is idle now
PASS 21h (6s)

== sanity-pcc test 21i: HSM release increase layout gen, should invalidate PCC-RO cache ========================================================== 18:21:48 (1746138108)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112043 s, 468 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d21i.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 0ce7158f-a752-4e94-ba6c-2dcf5a42cabc
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f21i.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=0ce7158f-a752-4e94-ba6c-2dcf5a42cabc (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d21i.sanity-pcc/d21i.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
/mnt/lustre/f21i.sanity-pcc: (0x00000009) exists archived, archive_id:2
Layout after HSM release /mnt/lustre/f21i.sanity-pcc:
/mnt/lustre/f21i.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    4
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x37
      lmm_fid:           [0x200003ab1:0x37:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       released
      lmm_layout_gen:    2
      lmm_stripe_offset: 0

  - lcme_id:             131075
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
PCC state /mnt/lustre/f21i.sanity-pcc:
file: /mnt/lustre/f21i.sanity-pcc, type: none
Layout after PCC-RO attach /mnt/lustre/f21i.sanity-pcc again:
/mnt/lustre/f21i.sanity-pcc
lmm_magic:         0x0BD10BD0
lmm_seq:           0x200003ab1
lmm_object_id:     0x37
lmm_fid:           [0x200003ab1:0x37:0x0]
lmm_stripe_count:  1
lmm_stripe_size:   4194304
lmm_pattern:       raid0
lmm_layout_gen:    5
lmm_stripe_offset: 0
	obdidx		 objid		 objid		 group
	     0	           278	        0x116	   0x240000400

PCC state:
file: /mnt/lustre/f21i.sanity-pcc, type: readonly, PCC_file: /d21i.sanity-pcc/3a86/0x200003ab1:0x37:0x0, open_count: 0, flags: 0
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d21i.sanity-pcc is idle now
PASS 21i (8s)

== sanity-pcc test 21j: PCC-RO for encrypted file ======== 18:21:56 (1746138116)

 SKIP: sanity-pcc test_21j client encryption not supported
SKIP 21j (3s)

== sanity-pcc test 22: Test RO-PCC attach for the HSM released file ========================================================== 18:21:59 (1746138119)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116629 s, 450 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d22.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 8b192440-bba4-48c4-a17e-cee18b46eef8
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f22.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=8b192440-bba4-48c4-a17e-cee18b46eef8 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d22.sanity-pcc/d22.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
Layout after PCC-RO attach /mnt/lustre/f22.sanity-pcc:
/mnt/lustre/f22.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    3
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x3c
      lmm_fid:           [0x200003ab1:0x3c:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    2
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x117:0x0] }

  - lcme_id:             131075
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    8594128896
      lhm_archive_ver:   9663677440
      lhm_archive_uuid:  ''
Layout after HSM release /mnt/lustre/f22.sanity-pcc:
/mnt/lustre/f22.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    4
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x3c
      lmm_fid:           [0x200003ab1:0x3c:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       released
      lmm_layout_gen:    2
      lmm_stripe_offset: 0

  - lcme_id:             131075
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
Layout after PCC-RO attach /mnt/lustre/f22.sanity-pcc again:
/mnt/lustre/f22.sanity-pcc
lmm_magic:         0x0BD10BD0
lmm_seq:           0x200003ab1
lmm_object_id:     0x3c
lmm_fid:           [0x200003ab1:0x3c:0x0]
lmm_stripe_count:  1
lmm_stripe_size:   4194304
lmm_pattern:       raid0
lmm_layout_gen:    5
lmm_stripe_offset: 1
	obdidx		 objid		 objid		 group
	     1	           279	        0x117	   0x280000400

Layout after PCC-RO detach /mnt/lustre/f22.sanity-pcc:
/mnt/lustre/f22.sanity-pcc
lmm_magic:         0x0BD10BD0
lmm_seq:           0x200003ab1
lmm_object_id:     0x3c
lmm_fid:           [0x200003ab1:0x3c:0x0]
lmm_stripe_count:  1
lmm_stripe_size:   4194304
lmm_pattern:       raid0
lmm_layout_gen:    5
lmm_stripe_offset: 1
	obdidx		 objid		 objid		 group
	     1	           279	        0x117	   0x280000400

Waiting 200s for 'SUCCEED'
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d22.sanity-pcc is idle now
PASS 22 (12s)

== sanity-pcc test 23: Test write/truncate/mmap-write invalidating RO-PCC caching ========================================================== 18:22:11 (1746138131)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.140456 s, 373 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d23.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 55a57d0a-73b6-47ed-8203-3fba244a2803
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f23.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=55a57d0a-73b6-47ed-8203-3fba244a2803 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d23.sanity-pcc/d23.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
Write should invalidate the RO-PCC cache:
/mnt/lustre/f23.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         wp
  lcm_layout_gen:    4
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x44
      lmm_fid:           [0x200003ab1:0x44:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x119:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init,stale
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,dirty,archived,0xb
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
PCC-RO attach '/mnt/lustre/f23.sanity-pcc' again:
/mnt/lustre/f23.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         wp,pccro
  lcm_layout_gen:    5
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x44
      lmm_fid:           [0x200003ab1:0x44:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x119:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
Truncate invalidate the RO-PCC cache:
/mnt/lustre/f23.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         wp
  lcm_layout_gen:    6
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x44
      lmm_fid:           [0x200003ab1:0x44:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x119:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,0x9
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
Finish trucate operation
Mmap write invalidates RO-PCC caching
/mnt/lustre/f23.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         wp
  lcm_layout_gen:    6
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x44
      lmm_fid:           [0x200003ab1:0x44:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x119:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,0x9
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
PCC-RO attach '/mnt/lustre/f23.sanity-pcc' again:
/mnt/lustre/f23.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         wp,pccro
  lcm_layout_gen:    7
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x44
      lmm_fid:           [0x200003ab1:0x44:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x119:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
Mmap write /mnt/lustre/f23.sanity-pcc via multiop
/mnt/lustre/f23.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         wp
  lcm_layout_gen:    8
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x200003ab1
      lmm_object_id:     0x44
      lmm_fid:           [0x200003ab1:0x44:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0x119:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,0x9
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d23.sanity-pcc is idle now
PASS 23 (7s)

== sanity-pcc test 24a: Test RO-PCC with non-root user === 18:22:18 (1746138138)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0976251 s, 537 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d24a.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: c9b000fd-e58b-4269-adc1-067e2e7dce26
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f24a.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=c9b000fd-e58b-4269-adc1-067e2e7dce26 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d24a.sanity-pcc/d24a.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.d24a.sanity-pcc/d24a.sanity-pcc
    hsmtool: posix_v2
    rwid: 2
    roid: 2
    flags: a0
    autocache: projid={100}
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d24a.sanity-pcc/f24a.sanity-pcc] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.010418 s, 98.3 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-r] [-i] [2] [/mnt/lustre/d24a.sanity-pcc/f24a.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d24a.sanity-pcc/f24a.sanity-pcc] [of=/dev/null] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00466365 s, 220 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [-k] [/mnt/lustre/d24a.sanity-pcc/f24a.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [touch] [/mnt/pcc.d24a.sanity-pcc/d24a.sanity-pcc/3af6/0x200003ab1:0x47:0x0]
touch: cannot touch '/mnt/pcc.d24a.sanity-pcc/d24a.sanity-pcc/3af6/0x200003ab1:0x47:0x0': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/pcc.d24a.sanity-pcc/d24a.sanity-pcc/3af6/0x200003ab1:0x47:0x0] [of=/dev/null] [bs=1024] [count=1]
dd: failed to open '/mnt/pcc.d24a.sanity-pcc/d24a.sanity-pcc/3af6/0x200003ab1:0x47:0x0': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/pcc.d24a.sanity-pcc/d24a.sanity-pcc/3af6/0x200003ab1:0x47:0x0] [bs=1024] [count=1]
dd: failed to open '/mnt/pcc.d24a.sanity-pcc/d24a.sanity-pcc/3af6/0x200003ab1:0x47:0x0': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-r] [-i] [2] [/mnt/lustre/d24a.sanity-pcc/f24a.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/d24a.sanity-pcc/f24a.sanity-pcc]
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d24a.sanity-pcc is idle now
PASS 24a (6s)

== sanity-pcc test 24b: General permission test for RO-PCC ========================================================== 18:22:24 (1746138144)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0953707 s, 550 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d24b.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: dd7b5d5b-de09-48e2-9bf9-8afaff557360
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f24b.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=dd7b5d5b-de09-48e2-9bf9-8afaff557360 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d24b.sanity-pcc/d24b.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00752304 s, 136 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d24b.sanity-pcc/f24b.sanity-pcc] [bs=1024] [count=1]
dd: failed to open '/mnt/lustre/d24b.sanity-pcc/f24b.sanity-pcc': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d24b.sanity-pcc/f24b.sanity-pcc] [of=/dev/null] [bs=1024] [count=1]
dd: failed to open '/mnt/lustre/d24b.sanity-pcc/f24b.sanity-pcc': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/dev/zero] [of=/mnt/lustre/d24b.sanity-pcc/f24b.sanity-pcc] [bs=1024] [count=1]
dd: failed to open '/mnt/lustre/d24b.sanity-pcc/f24b.sanity-pcc': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d24b.sanity-pcc/f24b.sanity-pcc] [of=/dev/null] [bs=1024] [count=1]
dd: failed to open '/mnt/lustre/d24b.sanity-pcc/f24b.sanity-pcc': Permission denied
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d24b.sanity-pcc/f24b.sanity-pcc] [of=/dev/null] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00360353 s, 284 kB/s
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/d24b.sanity-pcc/f24b.sanity-pcc]
detach: cannot detach '/mnt/lustre/d24b.sanity-pcc/f24b.sanity-pcc' from PCC: Operation not permitted
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/d24b.sanity-pcc/f24b.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [dd] [if=/mnt/lustre/d24b.sanity-pcc/f24b.sanity-pcc] [of=/dev/null] [bs=1024] [count=1]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00277624 s, 369 kB/s
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d24b.sanity-pcc is idle now
PASS 24b (6s)

== sanity-pcc test 25: Tolerate fake read failure for RO-PCC ========================================================== 18:22:30 (1746138150)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0905414 s, 579 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d25.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 4ff45516-1c1b-4fda-a3f2-e9c91e74b109
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f25.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=4ff45516-1c1b-4fda-a3f2-e9c91e74b109 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d25.sanity-pcc/d25.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
fail_loc=0x1411
fail_loc=0
fail_loc=0x1411
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d25.sanity-pcc is idle now
PASS 25 (6s)

== sanity-pcc test 26: Repeat the attach/detach when the file has multiple openers ========================================================== 18:22:36 (1746138156)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0984983 s, 532 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d26.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: d7436c49-2753-445d-9c92-b4498eab878a
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f26.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=d7436c49-2753-445d-9c92-b4498eab878a (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d26.sanity-pcc/d26.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
multiop /mnt/lustre/f26.sanity-pcc vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.2652
client oleg101-client.virtnet multiop_bg started multiop_pid=2663
Stopping multiop_pid=2663 (kill 2663 on oleg101-client.virtnet)
multiop /mnt/lustre/f26.sanity-pcc vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.2767
client oleg101-client.virtnet multiop_bg started multiop_pid=2777
Stopping multiop_pid=2777 (kill 2777 on oleg101-client.virtnet)
multiop /mnt/lustre/f26.sanity-pcc vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.2882
client oleg101-client.virtnet multiop_bg started multiop_pid=2892
Stopping multiop_pid=2892 (kill 2892 on oleg101-client.virtnet)
multiop /mnt/lustre/f26.sanity-pcc vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.2997
client oleg101-client.virtnet multiop_bg started multiop_pid=3007
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Stopping multiop_pid=3007 (kill 3007 on oleg101-client.virtnet)
multiop /mnt/lustre/f26.sanity-pcc vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.3148
client oleg101-client.virtnet multiop_bg started multiop_pid=3158
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Stopping multiop_pid=3158 (kill 3158 on oleg101-client.virtnet)
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d26.sanity-pcc is idle now
PASS 26 (27s)

== sanity-pcc test 27: Auto attach at open when the file has multiple openers ========================================================== 18:23:03 (1746138183)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104213 s, 503 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d27.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: cbbe0659-da0f-4d0f-bbea-ab9dcb21b778
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f27.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=cbbe0659-da0f-4d0f-bbea-ab9dcb21b778 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d27.sanity-pcc/d27.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
multiop /mnt/lustre/f27.sanity-pcc vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.4063
client oleg101-client.virtnet multiop_bg started multiop_pid=4074
Waiting 200s for 'SUCCEED'
Stopping multiop_pid=4074 (kill 4074 on oleg101-client.virtnet)
multiop /mnt/lustre/f27.sanity-pcc vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.4207
client oleg101-client.virtnet multiop_bg started multiop_pid=4217
ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa7df800.lru_size=clear
ldlm.namespaces.lustre-MDT0000-mdc-ffff88012d0da000.lru_size=clear
Waiting 200s for 'SUCCEED'
Stopping multiop_pid=4217 (kill 4217 on oleg101-client.virtnet)
multiop /mnt/lustre/f27.sanity-pcc vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.4350
client oleg101-client.virtnet multiop_bg started multiop_pid=4360
Stopping multiop_pid=4360 (kill 4360 on oleg101-client.virtnet)
multiop /mnt/lustre/f27.sanity-pcc vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.4469
client oleg101-client.virtnet multiop_bg started multiop_pid=4479
ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa7df800.lru_size=clear
ldlm.namespaces.lustre-MDT0000-mdc-ffff88012d0da000.lru_size=clear
Stopping multiop_pid=4479 (kill 4479 on oleg101-client.virtnet)
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d27.sanity-pcc is idle now
PASS 27 (25s)

== sanity-pcc test 28: RW-PCC attach should fail when the file has cluster-wide openers ========================================================== 18:23:28 (1746138208)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12033 s, 436 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d28.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: cf7dfeff-1454-4fc7-a2f9-29f8921c6f13
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f28.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=cf7dfeff-1454-4fc7-a2f9-29f8921c6f13 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d28.sanity-pcc/d28.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
multiop /mnt/lustre/f28.sanity-pcc vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.5338
client oleg101-client.virtnet multiop_bg started multiop_pid=5348
lfs pcc pcc: cannot get WRITE lease, ext 0 for [0x200003ab1:0x5f:0x0]: Device or resource busy (16)
lfs pcc pcc: cannot get lease: Device or resource busy (16)
attach: cannot attach '/mnt/lustre/f28.sanity-pcc' to PCC with attach ID '2': Device or resource busy
Stopping multiop_pid=5348 (kill 5348 on oleg101-client.virtnet)
multiop /mnt/lustre2/f28.sanity-pcc vO_c
TMPPIPE=/tmp/multiop_open_wait_pipe.7040
lfs pcc pcc: cannot get WRITE lease, ext 0 for [0x200003ab1:0x5f:0x0]: Device or resource busy (16)
lfs pcc pcc: cannot get lease: Device or resource busy (16)
attach: cannot attach '/mnt/lustre/f28.sanity-pcc' to PCC with attach ID '2': Device or resource busy
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d28.sanity-pcc is idle now
PASS 28 (10s)

== sanity-pcc test 29a: Auto readonly caching on RO-PCC backend for O_RDONLY open ========================================================== 18:23:38 (1746138218)
using SAVE_PROJECT_SUPPORTED=0
Stopping clients: oleg101-client.virtnet /mnt/lustre (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre opts:
Stopping clients: oleg101-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre2 opts:
Stopping /mnt/lustre-mds1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg101-server
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
Checking servers environments
Checking clients oleg101-client.virtnet environments
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory
loading modules on: 'oleg101-server'
oleg101-server: oleg101-server.virtnet: executing load_modules_local
oleg101-server: Loading modules from /home/green/git/lustre-release/lustre
oleg101-server: detected 4 online CPUs by sysfs
oleg101-server: MODOPTS_LIBCFS=
oleg101-server: Force libcfs to create 2 CPU partitions
Setup mgs, mdt, osts
Starting mds1: -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-MDT0000
Starting ost1: -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0000
Starting ost2: -o localrecov  lustre-ost2/ost2 /mnt/lustre-ost2
seq.cli-lustre-OST0001-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0001
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800b5871800.idle_timeout=debug
osc.lustre-OST0000-osc-ffff8800b5873800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800b5871800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800b5873800.idle_timeout=debug
disable quota as required
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117375 s, 447 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d29a.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: a49663ad-b559-455f-99b0-1a18ce555f8c
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f29a.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=a49663ad-b559-455f-99b0-1a18ce555f8c (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d29a.sanity-pcc/d29a.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.d29a.sanity-pcc/d29a.sanity-pcc
    hsmtool: posix_v2
    rwid: 2
    roid: 2
    flags: ae
    autocache: projid={100}
  100 P /mnt/lustre/d29a.sanity-pcc/f29a.sanity-pcc
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d29a.sanity-pcc is idle now
PASS 29a (84s)

== sanity-pcc test 29b: Auto PCC-RO attach in atomic_open ========================================================== 18:25:02 (1746138302)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128478 s, 408 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d29b.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: bfdd579b-8854-47c2-bf59-15bed3a6f190
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f29b.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=bfdd579b-8854-47c2-bf59-15bed3a6f190 (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d29b.sanity-pcc/d29b.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: fname={*.dat}
1+0 records in
1+0 records out
4096 bytes (4.1 kB) copied, 0.0114738 s, 357 kB/s
1+0 records in
1+0 records out
4096 bytes (4.1 kB) copied, 0.0207076 s, 198 kB/s
file: /mnt/lustre/myfile.dat, type: readonly, PCC_file: /d29b.sanity-pcc/4282/0x200004281:0x3:0x0, open_count: 0, flags: 0
1+0 records in
1+0 records out
4096 bytes (4.1 kB) copied, 0.00678178 s, 604 kB/s
vm.drop_caches = 3
1+0 records in
1+0 records out
4096 bytes (4.1 kB) copied, 0.00511487 s, 801 kB/s
file: /mnt/lustre/myfile.dat, type: readonly, PCC_file: /d29b.sanity-pcc/4282/0x200004281:0x3:0x0, open_count: 0, flags: 0
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d29b.sanity-pcc is idle now
PASS 29b (7s)

== sanity-pcc test 30: Test lctl pcc del command ========= 18:25:09 (1746138309)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138745 s, 378 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d30.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 0714f812-91fd-4703-b4d1-367b6aabb4d5
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f30.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=0714f812-91fd-4703-b4d1-367b6aabb4d5 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.d30.sanity-pcc/d30.sanity-pcc
    hsmtool: posix_v2
    rwid: 2
    roid: 2
    flags: 3e
    autocache: projid={100}
check file 0:'428b' under directory '/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc'
check file 0:'4289' under directory '/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc'
check file 0:'4287' under directory '/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc'
check file 1:'0x200004281:0xa:0x0' under directory '/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc/428b'
Handle the file: 0x200004281:0xa:0x0
Detach and remove the PCC cached file: 0x200004281:0xa:0x0
check file 1:'0x200004281:0x8:0x0' under directory '/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc/4289'
Handle the file: 0x200004281:0x8:0x0
Remove non-cached file: /mnt/pcc.d30.sanity-pcc/d30.sanity-pcc/4289/0x200004281:0x8:0x0 flags: 1
check file 1:'0x200004281:0x6:0x0' under directory '/mnt/pcc.d30.sanity-pcc/d30.sanity-pcc/4287'
Handle the file: 0x200004281:0x6:0x0
Detach and remove the PCC cached file: 0x200004281:0x6:0x0
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d30.sanity-pcc is idle now
PASS 30 (7s)

== sanity-pcc test 31: Test lctl pcc del command with --keep option ========================================================== 18:25:17 (1746138317)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10427 s, 503 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d31.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 706084de-e472-4456-b7ef-da13021e850f
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f31.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=706084de-e472-4456-b7ef-da13021e850f (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d31.sanity-pcc/d31.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.d31.sanity-pcc/d31.sanity-pcc
    hsmtool: posix_v2
    rwid: 2
    roid: 2
    flags: 30
    autocache: projid={100}
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d31.sanity-pcc is idle now
PASS 31 (7s)

== sanity-pcc test 32: Test for RO-PCC when PCC copy is deleted ========================================================== 18:25:24 (1746138324)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138038 s, 380 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d32.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 61f3b8da-1a97-4434-9e09-ed3c9248e052
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f32.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=61f3b8da-1a97-4434-9e09-ed3c9248e052 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d32.sanity-pcc/d32.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
multiop /mnt/lustre/f32.sanity-pcc vo_rc
TMPPIPE=/tmp/multiop_open_wait_pipe.11649
client oleg101-client.virtnet multiop_bg started multiop_pid=11659
Stopping multiop_pid=11659 (kill 11659 on oleg101-client.virtnet)
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d32.sanity-pcc is idle now
PASS 32 (14s)

== sanity-pcc test 33: Cache rule with comparator (>, =, <) for file size ========================================================== 18:25:38 (1746138338)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0944854 s, 555 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d33.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 567e9db4-72b0-4bf6-a919-c49384eb6901
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f33.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=567e9db4-72b0-4bf6-a919-c49384eb6901 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d33.sanity-pcc/d33.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.d33.sanity-pcc/d33.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: fname={*.doc}&size<{1M}
file: /mnt/lustre/myfile.doc, type: none
file: /mnt/lustre/myfile.doc, type: none
Cleanup PCC backend on /mnt/lustre
pcc:
  -
    pccpath: /mnt/pcc.d33.sanity-pcc/d33.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 5
    flags: ae
    autocache: fname={*.doc}&size<{5M}&size>{3M}
Cleanup PCC backend on /mnt/lustre
pcc:
  -
    pccpath: /mnt/pcc.d33.sanity-pcc/d33.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 5
    flags: ae
    autocache: fname={*.doc}&size={5M 3M}
Cleanup PCC backend on /mnt/lustre
Cleanup PCC backend on /mnt/lustre
Cleanup PCC backend on /mnt/lustre
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d33.sanity-pcc is idle now
PASS 33 (9s)

== sanity-pcc test 34: Cache rule with comparator (>, <) for Project ID range ========================================================== 18:25:47 (1746138347)
using SAVE_PROJECT_SUPPORTED=0
Stopping clients: oleg101-client.virtnet /mnt/lustre (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre opts:
Stopping clients: oleg101-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre2 opts:
Stopping /mnt/lustre-mds1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg101-server
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
Checking servers environments
Checking clients oleg101-client.virtnet environments
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory
libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory
loading modules on: 'oleg101-server'
oleg101-server: oleg101-server.virtnet: executing load_modules_local
oleg101-server: Loading modules from /home/green/git/lustre-release/lustre
oleg101-server: detected 4 online CPUs by sysfs
oleg101-server: MODOPTS_LIBCFS=
oleg101-server: Force libcfs to create 2 CPU partitions
Setup mgs, mdt, osts
Starting mds1: -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-MDT0000
Starting ost1: -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0000
Starting ost2: -o localrecov  lustre-ost2/ost2 /mnt/lustre-ost2
seq.cli-lustre-OST0001-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0001
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800b5876000.idle_timeout=debug
osc.lustre-OST0000-osc-ffff8800b638e800.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800b5876000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800b638e800.idle_timeout=debug
disable quota as required
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111174 s, 472 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d34.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: b1513ff4-bc2c-4388-8ac2-1b329ac455b3
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f34.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=b1513ff4-bc2c-4388-8ac2-1b329ac455b3 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d34.sanity-pcc/d34.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
pcc:
  -
    pccpath: /mnt/pcc.d34.sanity-pcc/d34.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 5
    flags: ae
    autocache: projid>{100}
   99 - /mnt/lustre/f34.sanity-pcc
  101 - /mnt/lustre/f34.sanity-pcc
Cleanup PCC backend on /mnt/lustre
pcc:
  -
    pccpath: /mnt/pcc.d34.sanity-pcc/d34.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 5
    flags: ae
    autocache: projid<{100}
  102 - /mnt/lustre/f34.sanity-pcc
   99 - /mnt/lustre/f34.sanity-pcc
Cleanup PCC backend on /mnt/lustre
pcc:
  -
    pccpath: /mnt/pcc.d34.sanity-pcc/d34.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 5
    flags: ae
    autocache: projid<{120}&projid>{110}
  105 - /mnt/lustre/f34.sanity-pcc
  121 - /mnt/lustre/f34.sanity-pcc
  115 - /mnt/lustre/f34.sanity-pcc
Cleanup PCC backend on /mnt/lustre
Cleanup PCC backend on /mnt/lustre
Cleanup PCC backend on /mnt/lustre
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d34.sanity-pcc is idle now
PASS 34 (69s)

== sanity-pcc test 35: mmap fault test =================== 18:26:57 (1746138417)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.186412 s, 281 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d35.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: a024d8e0-ff7d-4132-8c68-40bc9d748731
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f35.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=a024d8e0-ff7d-4132-8c68-40bc9d748731 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d35.sanity-pcc/d35.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d35.sanity-pcc is idle now
PASS 35 (7s)

== sanity-pcc test 36a: Stale RW-PCC copy should be deleted after remove the PCC backend ========================================================== 18:27:04 (1746138424)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101323 s, 517 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d36a.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 550fc514-5c4e-448e-9bf2-55046ff2f6c7
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f36a.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=550fc514-5c4e-448e-9bf2-55046ff2f6c7 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d36a.sanity-pcc/d36a.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
check file 0:'4a52' under directory '/mnt/pcc.d36a.sanity-pcc/d36a.sanity-pcc'
check file 1:'0x200004a51:0x3:0x0' under directory '/mnt/pcc.d36a.sanity-pcc/d36a.sanity-pcc/4a52'
Handle the file: 0x200004a51:0x3:0x0
Remove non-cached file: /mnt/pcc.d36a.sanity-pcc/d36a.sanity-pcc/4a52/0x200004a51:0x3:0x0 flags: 1
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d36a.sanity-pcc is idle now
PASS 36a (6s)

== sanity-pcc test 36b: Stale RO-PCC copy should be deleted after remove the PCC backend ========================================================== 18:27:10 (1746138430)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.220763 s, 237 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d36b.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: d1d1a6c3-09ec-4392-a87e-439c5bfecdc1
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f36b.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=d1d1a6c3-09ec-4392-a87e-439c5bfecdc1 (extents) (64bit) (large files) (huge files)
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d36b.sanity-pcc/d36b.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
check file 0:'4a54' under directory '/mnt/pcc.d36b.sanity-pcc/d36b.sanity-pcc'
check file 1:'0x200004a51:0x5:0x0' under directory '/mnt/pcc.d36b.sanity-pcc/d36b.sanity-pcc/4a54'
Handle the file: 0x200004a51:0x5:0x0
Remove non-cached file: /mnt/pcc.d36b.sanity-pcc/d36b.sanity-pcc/4a54/0x200004a51:0x5:0x0 flags: 1
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d36b.sanity-pcc is idle now
PASS 36b (7s)

== sanity-pcc test 37: Multiple readers on a shared file with PCC-RO mode ========================================================== 18:27:17 (1746138437)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.135152 s, 388 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d37.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 1e98734c-0079-493f-996f-4bceba45c6f3
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f37.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=1e98734c-0079-493f-996f-4bceba45c6f3 (extents) (64bit) (large files) (huge files)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0843469 s, 622 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
/tmp/f37.sanity-pcc on /mnt/pcc.d37.sanity-pcc type ext4 (rw,relatime,quota,usrquota,grpquota,data=ordered)
umount: /mnt/pcc.d37.sanity-pcc.2: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
/tmp/f37.sanity-pcc on /mnt/pcc.d37.sanity-pcc type ext4 (rw,relatime,quota,usrquota,grpquota,data=ordered)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: c1eb2cc5-a85b-4498-85a4-f9d35e500091
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f37.sanity-pcc.2: Linux rev 1.0 ext4 filesystem data, UUID=c1eb2cc5-a85b-4498-85a4-f9d35e500091 (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d37.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: a0
    autocache: projid={2}
pcc:
  -
    pccpath: /mnt/pcc.d37.sanity-pcc.2
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: a0
    autocache: projid={2}
file: /mnt/lustre/d37.sanity-pcc/f37.sanity-pcc, type: readonly, PCC_file: /4a56/0x200004a51:0x7:0x0, open_count: 0, flags: 0
file: /mnt/lustre2/d37.sanity-pcc/f37.sanity-pcc, type: readonly, PCC_file: /4a56/0x200004a51:0x7:0x0, open_count: 0, flags: 0
/mnt/pcc.d37.sanity-pcc.2 is idle now
/mnt/pcc.d37.sanity-pcc is idle now
PASS 37 (10s)

== sanity-pcc test 38: Verify LFS pcc state does not trigger prefetch for auto PCC-RO ========================================================== 18:27:28 (1746138448)
using SAVE_PROJECT_SUPPORTED=0
Stopping clients: oleg101-client.virtnet /mnt/lustre (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre opts:
Stopping clients: oleg101-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre2 opts:
Stopping /mnt/lustre-mds1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg101-server
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
Checking servers environments
Checking clients oleg101-client.virtnet environments
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
loading modules on: 'oleg101-server'
oleg101-server: oleg101-server.virtnet: executing load_modules_local
oleg101-server: Loading modules from /home/green/git/lustre-release/lustre
oleg101-server: detected 4 online CPUs by sysfs
oleg101-server: MODOPTS_LIBCFS=
oleg101-server: Force libcfs to create 2 CPU partitions
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory
oleg101-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory
Setup mgs, mdt, osts
Starting mds1: -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-MDT0000
Starting ost1: -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0000
Starting ost2: -o localrecov  lustre-ost2/ost2 /mnt/lustre-ost2
seq.cli-lustre-OST0001-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0001
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800a1783000.idle_timeout=debug
osc.lustre-OST0000-osc-ffff8800b6cf0000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800a1783000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800b6cf0000.idle_timeout=debug
disable quota as required
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112748 s, 465 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d38.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 8b48497f-9bd8-49fa-843f-492bac077fb9
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f38.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=8b48497f-9bd8-49fa-843f-492bac077fb9 (extents) (64bit) (large files) (huge files)
file: /mnt/lustre/d38.sanity-pcc/f38.sanity-pcc, type: none
QQQQQ
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d38.sanity-pcc is idle now
PASS 38 (86s)

== sanity-pcc test 39: Test Project quota on loop PCC device ========================================================== 18:28:54 (1746138534)

 SKIP: sanity-pcc test_39 Not support project quota on local filesystem
SKIP 39 (2s)

== sanity-pcc test 40: Test async open attach in the background for PCC-RO file ========================================================== 18:28:56 (1746138536)
using SAVE_PROJECT_SUPPORTED=0
Stopping clients: oleg101-client.virtnet /mnt/lustre (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre opts:
Stopping clients: oleg101-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre2 opts:
Stopping /mnt/lustre-mds1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg101-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg101-server
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
Checking servers environments
Checking clients oleg101-client.virtnet environments
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory
libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory
libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory
loading modules on: 'oleg101-server'
oleg101-server: oleg101-server.virtnet: executing load_modules_local
oleg101-server: Loading modules from /home/green/git/lustre-release/lustre
oleg101-server: detected 4 online CPUs by sysfs
oleg101-server: MODOPTS_LIBCFS=
oleg101-server: Force libcfs to create 2 CPU partitions
Setup mgs, mdt, osts
Starting mds1: -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-MDT0000
Starting ost1: -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0000
Starting ost2: -o localrecov  lustre-ost2/ost2 /mnt/lustre-ost2
seq.cli-lustre-OST0001-super.width=65536
oleg101-server: oleg101-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg101-client: oleg101-server: ssh exited with exit code 1
Started lustre-OST0001
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Starting client: oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Starting client oleg101-client.virtnet:  -o user_xattr,flock 192.168.201.101@tcp:/lustre /mnt/lustre2
Started clients oleg101-client.virtnet: 
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
Using TIMEOUT=20
osc.lustre-OST0000-osc-ffff8800a99f2000.idle_timeout=debug
osc.lustre-OST0000-osc-ffff8800b6371000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800a99f2000.idle_timeout=debug
osc.lustre-OST0001-osc-ffff8800b6371000.idle_timeout=debug
disable quota as required
200+0 records in
200+0 records out
209715200 bytes (210 MB) copied, 0.484957 s, 432 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d40.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:   1024/204800             done                            
Creating filesystem with 204800 1k blocks and 51200 inodes
Filesystem UUID: 8858fe7d-0032-41b2-8527-0b6efd5fcbbd
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345, 73729

Allocating group tables:  0/25     done                            
Writing inode tables:  0/25     done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information:  0/25     done

/tmp/f40.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=8858fe7d-0032-41b2-8527-0b6efd5fcbbd (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d40.sanity-pcc/d40.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={100}
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.901645 s, 58.1 MB/s
  100 - /mnt/lustre/d40.sanity-pcc/f40.sanity-pcc
file: /mnt/lustre/d40.sanity-pcc/f40.sanity-pcc, type: none
ldlm.namespaces.lustre-OST0000-osc-ffff8800a99f2000.lru_size=clear
ldlm.namespaces.lustre-OST0000-osc-ffff8800b6371000.lru_size=clear
ldlm.namespaces.lustre-OST0001-osc-ffff8800a99f2000.lru_size=clear
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6371000.lru_size=clear
osc.lustre-OST0000-osc-ffff8800a99f2000.stats=clear
osc.lustre-OST0000-osc-ffff8800b6371000.stats=clear
osc.lustre-OST0001-osc-ffff8800a99f2000.stats=clear
osc.lustre-OST0001-osc-ffff8800b6371000.stats=clear
fail_val=1
fail_loc=0x214
llite.lustre-ffff8800a99f2000.pcc_async_threshold=5MB
llite.lustre-ffff8800b6371000.pcc_async_threshold=5MB
Test open attach with pcc_async_threshold=5MB
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 2.15777 s, 486 kB/s
Before: 0 After: 3
file: /mnt/lustre/d40.sanity-pcc/f40.sanity-pcc, type: none, flags: attaching
/home/green/git/lustre-release/lustre/utils/lfs pcc state /mnt/lustre/d40.sanity-pcc/f40.sanity-pcc | grep -E -c 'type: readonly'
Waiting 50s for '1'
Updated after 7s: want '1' got '1'
file: /mnt/lustre/d40.sanity-pcc/f40.sanity-pcc, type: none
llite.lustre-ffff8800a99f2000.pcc_async_threshold=1G
llite.lustre-ffff8800b6371000.pcc_async_threshold=1G
ldlm.namespaces.lustre-OST0000-osc-ffff8800a99f2000.lru_size=clear
ldlm.namespaces.lustre-OST0000-osc-ffff8800b6371000.lru_size=clear
ldlm.namespaces.lustre-OST0001-osc-ffff8800a99f2000.lru_size=clear
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6371000.lru_size=clear
osc.lustre-OST0000-osc-ffff8800a99f2000.stats=clear
osc.lustre-OST0000-osc-ffff8800b6371000.stats=clear
osc.lustre-OST0001-osc-ffff8800a99f2000.stats=clear
osc.lustre-OST0001-osc-ffff8800b6371000.stats=clear
Test open attach with async_threshold=1G
file: /mnt/lustre/d40.sanity-pcc/f40.sanity-pcc, type: readonly, PCC_file: /d40.sanity-pcc/59f3/0x2000059f1:0x2:0x0, open_count: 0, flags: 0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0108395 s, 96.7 MB/s
Before: 50 After: 50
Time1: 2 Time2: 10
file: /mnt/lustre/d40.sanity-pcc/f40.sanity-pcc, type: none
llite.lustre-ffff8800a99f2000.pcc_async_threshold=5MB
llite.lustre-ffff8800b6371000.pcc_async_threshold=5MB
ldlm.namespaces.lustre-OST0000-osc-ffff8800a99f2000.lru_size=clear
ldlm.namespaces.lustre-OST0000-osc-ffff8800b6371000.lru_size=clear
ldlm.namespaces.lustre-OST0001-osc-ffff8800a99f2000.lru_size=clear
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6371000.lru_size=clear
Read 1MB data with async_threshold=5MB
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 1.05237 s, 996 kB/s
/home/green/git/lustre-release/lustre/utils/lfs pcc state /mnt/lustre/d40.sanity-pcc/f40.sanity-pcc | grep -E -c 'type: readonly'
Waiting 50s for '1'
Updated after 9s: want '1' got '1'
file: /mnt/lustre/d40.sanity-pcc/f40.sanity-pcc, type: none
llite.lustre-ffff8800a99f2000.pcc_async_threshold=1G
llite.lustre-ffff8800b6371000.pcc_async_threshold=1G
ldlm.namespaces.lustre-OST0000-osc-ffff8800a99f2000.lru_size=clear
ldlm.namespaces.lustre-OST0000-osc-ffff8800b6371000.lru_size=clear
ldlm.namespaces.lustre-OST0001-osc-ffff8800a99f2000.lru_size=clear
ldlm.namespaces.lustre-OST0001-osc-ffff8800b6371000.lru_size=clear
Read 1MB data with async_threshold=1G
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00985836 s, 106 MB/s
Time1: 2 Time2: 10
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d40.sanity-pcc is idle now
PASS 40 (129s)

== sanity-pcc test 41: Test mtime rule for PCC-RO open attach with O_RDONLY mode ========================================================== 18:31:06 (1746138666)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0952228 s, 551 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d41.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: c9bb8120-217d-4158-bd6a-a96e1490163c
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f41.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=c9bb8120-217d-4158-bd6a-a96e1490163c (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d41.sanity-pcc/d41.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: mtime>{1m}
pcc_ro_data
pcc_ro_data
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d41.sanity-pcc is idle now
PASS 41 (6s)

== sanity-pcc test 42: PCC attach without attach ID specified ========================================================== 18:31:12 (1746138672)
60+0 records in
60+0 records out
62914560 bytes (63 MB) copied, 0.215733 s, 292 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d42.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/61440           done                            
Creating filesystem with 61440 1k blocks and 15360 inodes
Filesystem UUID: 167538c9-d335-4dc6-b385-fd606d65327c
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345

Allocating group tables: 0/8   done                            
Writing inode tables: 0/8   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/8   done

/tmp/f42.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=167538c9-d335-4dc6-b385-fd606d65327c (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d42.sanity-pcc/d42.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={100}
file: /mnt/lustre/f42.sanity-pcc, type: readonly, PCC_file: /d42.sanity-pcc/59f5/0x2000059f1:0x4:0x0, open_count: 0, flags: 0
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d42.sanity-pcc is idle now
PASS 42 (8s)

== sanity-pcc test 43: Auto attach at open() should add capacity owner check ========================================================== 18:31:20 (1746138680)
60+0 records in
60+0 records out
62914560 bytes (63 MB) copied, 0.139114 s, 452 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d43.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/61440           done                            
Creating filesystem with 61440 1k blocks and 15360 inodes
Filesystem UUID: c7c70a69-8898-4f64-867b-f21945b7d305
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345

Allocating group tables: 0/8   done                            
Writing inode tables: 0/8   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/8   done

/tmp/f43.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=c7c70a69-8898-4f64-867b-f21945b7d305 (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d43.sanity-pcc/d43.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: size<{100M}
file: /mnt/lustre/f43.sanity-pcc, type: none
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-r] [/mnt/lustre/f43.sanity-pcc]
lfs pcc pcc: cannot attach the file to PCC with ID 0 failed: Operation not permitted (1)
attach: cannot attach '/mnt/lustre/f43.sanity-pcc' to PCC with attach ID '0': Operation not permitted
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [state] [/mnt/lustre/f43.sanity-pcc]
file: /mnt/lustre/f43.sanity-pcc, type: none
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d43.sanity-pcc is idle now
PASS 43 (7s)

== sanity-pcc test 44: Verify valid auto attach without re-fetching the whole files ========================================================== 18:31:27 (1746138687)
60+0 records in
60+0 records out
62914560 bytes (63 MB) copied, 0.150278 s, 419 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d44.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/61440           done                            
Creating filesystem with 61440 1k blocks and 15360 inodes
Filesystem UUID: 6c8c922e-023b-43fa-a473-0102d0fce1c6
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345

Allocating group tables: 0/8   done                            
Writing inode tables: 0/8   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/8   done

/tmp/f44.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=6c8c922e-023b-43fa-a473-0102d0fce1c6 (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d44.sanity-pcc/d44.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={0}
llite.lustre-ffff8800a99f2000.pcc_async_threshold=0
llite.lustre-ffff8800b6371000.pcc_async_threshold=0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.883186 s, 59.4 MB/s
/mnt/lustre/f44.sanity-pcc
lmm_magic:         0x0BD10BD0
lmm_seq:           0x2000059f1
lmm_object_id:     0x6
lmm_fid:           [0x2000059f1:0x6:0x0]
lmm_stripe_count:  1
lmm_stripe_size:   4194304
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 1
	obdidx		 objid		 objid		 group
	     1	           420	        0x1a4	   0x280000400

50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0807841 s, 649 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.159402 s, 329 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.180089 s, 291 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.242147 s, 217 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0858674 s, 611 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.227985 s, 230 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.202627 s, 259 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122759 s, 427 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.249126 s, 210 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.241228 s, 217 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.281935 s, 186 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.270699 s, 194 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12673 s, 414 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.291071 s, 180 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.572279 s, 91.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.873278 s, 60.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.565034 s, 92.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.829394 s, 63.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.827092 s, 63.4 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.696173 s, 75.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.865684 s, 60.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.33422 s, 157 MB/s
52428800 bytes (52 MB) copied, 0.695282 s, 75.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.878081 s, 59.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.728554 s, 72.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.741295 s, 70.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.894512 s, 58.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.889681 s, 58.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.762363 s, 68.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.74088 s, 70.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0318243 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0220151 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.8144 s, 64.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0199256 s, 2.6 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0716485 s, 732 MB/s
52428800 bytes (52 MB) copied, 0.0512492 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0165071 s, 3.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0616955 s, 850 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0507218 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0891792 s, 588 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.133418 s, 393 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0855722 s, 613 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121502 s, 432 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13933 s, 376 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151279 s, 347 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0521283 s, 1.0 GB/s
52428800 bytes (52 MB) copied, 0.155047 s, 338 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.08157 s, 643 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0371043 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.22042 s, 238 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.16703 s, 314 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.190829 s, 275 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.180739 s, 290 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0827426 s, 634 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0728634 s, 720 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0918028 s, 571 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0324663 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0583808 s, 898 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119796 s, 438 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0356772 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0794062 s, 660 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108121 s, 485 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103137 s, 508 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0871637 s, 601 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0397327 s, 1.3 GB/s
, 0.0417774 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0489888 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0981203 s, 534 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119595 s, 438 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0868058 s, 604 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0357499 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0703417 s, 745 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107266 s, 489 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106013 s, 495 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0799215 s, 656 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0795925 s, 659 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12585 s, 417 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.203585 s, 258 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.150426 s, 349 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.187601 s, 279 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.048691 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.185988 s, 282 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0668122 s, 785 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.166857 s, 314 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0904471 s, 580 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0422623 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0222565 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.177624 s, 295 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.170091 s, 308 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.23267 s, 225 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0695985 s, 753 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0480108 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.166059 s, 316 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0395276 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.199599 s, 263 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0660989 s, 793 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0314753 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.241689 s, 217 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.219421 s, 239 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.267868 s, 196 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0433228 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0883654 s, 593 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0582655 s, 900 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0197045 s, 2.7 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128199 s, 409 MB/s
52428800 bytes (52 MB) copied, 0.108849 s, 482 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0674392 s, 777 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.064153 s, 817 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0828347 s, 633 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0799299 s, 656 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120562 s, 435 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137332 s, 382 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0424518 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0683049 s, 768 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0540097 s, 971 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.055473 s, 945 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0716154 s, 732 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0922934 s, 568 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0602822 s, 870 MB/s
52428800 bytes (52 MB) copied, 0.0345495 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0236446 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10275 s, 510 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0944096 s, 555 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0323163 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0546304 s, 960 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.036208 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0495721 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0629579 s, 833 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0638224 s, 821 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0772696 s, 679 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105156 s, 499 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0761913 s, 688 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.125279 s, 418 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.148621 s, 353 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117047 s, 448 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101288 s, 518 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145557 s, 360 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.184902 s, 284 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0193021 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0584797 s, 897 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0255696 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107901 s, 486 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0507455 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0411131 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0447656 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0651481 s, 805 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.051791 s, 1.0 GB/s
52428800 bytes (52 MB) copied, 0.0392613 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0701768 s, 747 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0203167 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0810126 s, 647 MB/s
52428800 bytes (52 MB) copied, 0.101321 s, 517 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0496034 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0380087 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0881507 s, 595 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0789849 s, 664 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101034 s, 519 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0532916 s, 984 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0924658 s, 567 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0520327 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0473275 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0462156 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468671 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0478505 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0533847 s, 982 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0658017 s, 797 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.066301 s, 791 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0444404 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0685264 s, 765 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0745664 s, 703 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0186232 s, 2.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0600085 s, 874 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0702416 s, 746 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0618892 s, 847 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0319433 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0544988 s, 962 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0741771 s, 707 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0608755 s, 861 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0952867 s, 550 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.140346 s, 374 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13106 s, 400 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0466819 s, 1.1 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0619094 s, 847 MB/s
52428800 bytes (52 MB) copied, 0.0705779 s, 743 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0789603 s, 664 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0841246 s, 623 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.061245 s, 856 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.06309 s, 831 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0704049 s, 745 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0657999 s, 797 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0597291 s, 878 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0775964 s, 676 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0650632 s, 806 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0966198 s, 543 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0419092 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0570821 s, 918 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0602092 s, 871 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0525204 s, 998 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.208524 s, 251 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0844662 s, 621 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0301498 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0672497 s, 780 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0558282 s, 939 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.123227 s, 425 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115185 s, 455 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108471 s, 483 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.20357 s, 258 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117275 s, 447 MB/s
, 0.0566488 s, 926 MB/s
52428800 bytes (52 MB) copied, 0.0851055 s, 616 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104369 s, 502 MB/s
52428800 bytes (52 MB) copied, 0.0993247 s, 528 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0202486 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.20278 s, 259 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.173365 s, 302 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.09854 s, 532 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138759 s, 378 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0625451 s, 838 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0668611 s, 784 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106131 s, 494 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.140595 s, 373 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.09675 s, 542 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0547657 s, 957 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.146473 s, 358 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.205467 s, 255 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.129613 s, 405 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0643087 s, 815 MB/s
52428800 bytes (52 MB) copied, 0.0651997 s, 804 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0297523 s, 1.8 GB/s
52428800 bytes (52 MB) copied, 0.144592 s, 363 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.141058 s, 372 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142868 s, 367 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.097137 s, 540 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0796836 s, 658 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151704 s, 346 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131531 s, 399 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0709295 s, 739 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0759391 s, 690 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0920742 s, 569 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0985085 s, 532 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.148857 s, 352 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108545 s, 483 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145404 s, 361 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382782 s, 1.4 GB/s
52428800 bytes (52 MB) copied, 0.0471071 s, 1.1 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.162488 s, 323 MB/s
52428800 bytes (52 MB) copied, 0.0905218 s, 579 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0567554 s, 924 MB/s
52428800 bytes (52 MB) copied, 0.0399468 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0945455 s, 555 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0449396 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.140571 s, 373 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0354979 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0811708 s, 646 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.057249 s, 916 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0319052 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.039395 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0526814 s, 995 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287889 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0404541 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.072586 s, 722 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0432082 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370442 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0662219 s, 792 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0678113 s, 773 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0937962 s, 559 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117588 s, 446 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0819326 s, 640 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0408507 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0587768 s, 892 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0517062 s, 1.0 GB/s
52428800 bytes (52 MB) copied, 0.077111 s, 680 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0660424 s, 794 MB/s
, 0.0544898 s, 962 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0191982 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0425518 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0555952 s, 943 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0485597 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0802435 s, 653 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0512986 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0422587 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0697683 s, 751 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0423868 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0665145 s, 788 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0549053 s, 955 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0873059 s, 601 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0942143 s, 556 MB/s
52428800 bytes (52 MB) copied, 0.079411 s, 660 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0714925 s, 733 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0432689 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0818725 s, 640 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0453542 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0910185 s, 576 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0378328 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0486977 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.035342 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0216888 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.032539 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0414685 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0859353 s, 610 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0909984 s, 576 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0748555 s, 700 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0278059 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0218195 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.033637 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0688135 s, 762 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0668285 s, 785 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0415621 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0615256 s, 852 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0591612 s, 886 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0455437 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0490583 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0456921 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0782686 s, 670 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0838749 s, 625 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0350103 s, 1.5 GB/s
52428800 bytes (52 MB) copied, 0.0822361 s, 638 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0593178 s, 884 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0663438 s, 790 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0523224 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131909 s, 397 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149663 s, 350 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.226363 s, 232 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112358 s, 467 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.03341 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0697865 s, 751 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0832443 s, 630 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0796666 s, 658 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0436405 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0546036 s, 960 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0597663 s, 877 MB/s
, 0.0396511 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0670803 s, 782 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0483494 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0203456 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0527267 s, 994 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0418821 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0577622 s, 908 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0542178 s, 967 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0438185 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.052167 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382764 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0270356 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0436986 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0492385 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0513578 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0771611 s, 679 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0837626 s, 626 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107366 s, 488 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0953535 s, 550 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0966928 s, 542 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138835 s, 378 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142193 s, 369 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124247 s, 422 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.168012 s, 312 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137353 s, 382 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.18296 s, 287 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0379717 s, 1.4 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.060322 s, 869 MB/s
, 0.0603427 s, 869 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0982922 s, 533 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0567801 s, 923 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0888887 s, 590 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116398 s, 450 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0922377 s, 568 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0649799 s, 807 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0390471 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0891074 s, 588 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112758 s, 465 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0559513 s, 937 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0326503 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468528 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0956482 s, 548 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.105326 s, 498 MB/s
, 0.0310502 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107182 s, 489 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0556047 s, 943 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0639769 s, 819 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112418 s, 466 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0243797 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522778 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0597649 s, 877 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0765063 s, 685 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0427996 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0514108 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0900791 s, 582 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116054 s, 452 MB/s
, 0.0915079 s, 573 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109632 s, 478 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0583492 s, 899 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0563255 s, 931 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0632111 s, 829 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0766208 s, 684 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0717923 s, 730 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109276 s, 480 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0718811 s, 729 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469091 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0421619 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0788016 s, 665 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105784 s, 496 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0968602 s, 541 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0527486 s, 994 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0750006 s, 699 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0763698 s, 687 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0945964 s, 554 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0726823 s, 721 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0954504 s, 549 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106512 s, 492 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0961996 s, 545 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0783407 s, 669 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0550271 s, 953 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0403867 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0289695 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.146581 s, 358 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.197563 s, 265 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0432397 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376949 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0510749 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0783601 s, 669 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308067 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0348092 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0998843 s, 525 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0353703 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0623063 s, 841 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0463313 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0587328 s, 893 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0355394 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468345 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0394426 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0440537 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0294525 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0885788 s, 592 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0485834 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0507226 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0777652 s, 674 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0421169 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0735622 s, 713 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0898251 s, 584 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0753712 s, 696 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306392 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0900531 s, 582 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101381 s, 517 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522744 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0758903 s, 691 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0650351 s, 806 MB/s
, 0.103288 s, 508 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0609494 s, 860 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0749453 s, 700 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0480007 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0916919 s, 572 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0673503 s, 778 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0900096 s, 582 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0464606 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0642382 s, 816 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0562086 s, 933 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0619965 s, 846 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.044503 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0937845 s, 559 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0199759 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0409704 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0320188 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0400999 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0564518 s, 929 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0593401 s, 884 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0530823 s, 988 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0197671 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115798 s, 453 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0255223 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306938 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0620542 s, 845 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0298091 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0232734 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0763173 s, 687 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0602022 s, 871 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0466628 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0589527 s, 889 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0713088 s, 735 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.038967 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0852311 s, 615 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0385026 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0430057 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0847647 s, 619 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0548811 s, 955 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0752412 s, 697 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0507529 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.084264 s, 622 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101129 s, 518 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10516 s, 499 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0436643 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0623021 s, 842 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0730426 s, 718 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0873756 s, 600 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308777 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0816351 s, 642 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0753464 s, 696 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0427173 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0610268 s, 859 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0512411 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0640235 s, 819 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0813199 s, 645 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0505413 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0681742 s, 769 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107331 s, 488 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0870729 s, 602 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0707711 s, 741 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0541944 s, 967 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0666119 s, 787 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0393612 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0270943 s, 1.9 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0384788 s, 1.4 GB/s
, 0.0479409 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0202769 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0514147 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0560568 s, 935 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0573289 s, 915 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0715823 s, 732 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0846005 s, 620 MB/s
, 0.127731 s, 410 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115881 s, 452 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0628608 s, 834 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0926289 s, 566 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113268 s, 463 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0329502 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151114 s, 347 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0747798 s, 701 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.141285 s, 371 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0359952 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0435358 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0335444 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0886958 s, 591 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105153 s, 499 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137759 s, 381 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0682533 s, 768 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0635478 s, 825 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0436697 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0737562 s, 711 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0931044 s, 563 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0288545 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0332538 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0758613 s, 691 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0673033 s, 779 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0702029 s, 747 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0625308 s, 838 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0843945 s, 621 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0613571 s, 854 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0946847 s, 554 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0367777 s, 1.4 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0336317 s, 1.6 GB/s
, 0.0740337 s, 708 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0680424 s, 771 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0354104 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0481468 s, 1.1 GB/s
, 0.100951 s, 519 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0420555 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0384071 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0510296 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0551054 s, 951 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0741941 s, 707 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0392831 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0502589 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0221728 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0513723 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0316647 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287252 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0245513 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.047029 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0294993 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0386488 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0298848 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0682537 s, 768 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0849701 s, 617 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0525478 s, 998 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0695114 s, 754 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0607967 s, 862 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0757364 s, 692 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0863599 s, 607 MB/s
52428800 bytes (52 MB) copied, 0.070635 s, 742 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.037684 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360532 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0429704 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0649376 s, 807 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0744741 s, 704 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0688865 s, 761 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0243587 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0300902 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0432689 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287742 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0399517 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0351422 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0295303 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0217868 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.051168 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0571386 s, 918 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0457363 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0345068 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0584243 s, 897 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0603578 s, 869 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0265525 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0346228 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306982 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0410095 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0491479 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0470585 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0777396 s, 674 MB/s
, 0.0692041 s, 758 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.074788 s, 701 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0729927 s, 718 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336724 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0707168 s, 741 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0461551 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0414414 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0944512 s, 555 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0523426 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0598315 s, 876 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.040884 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0502146 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0811117 s, 646 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.073067 s, 718 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0361048 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0391287 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0476553 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0234948 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0638526 s, 821 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0428779 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0290379 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.027339 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0836318 s, 627 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.125954 s, 416 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.051565 s, 1.0 GB/s
52428800 bytes (52 MB) copied, 0.0613713 s, 854 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136473 s, 384 MB/s
52428800 bytes (52 MB) copied, 0.0853282 s, 614 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0959892 s, 546 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.177572 s, 295 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0484172 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0508949 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0599027 s, 875 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151438 s, 346 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0544912 s, 962 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0524907 s, 999 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0464235 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0615767 s, 851 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0514462 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0652021 s, 804 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0345589 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311428 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0681765 s, 769 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0292656 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105122 s, 499 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0873817 s, 600 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0705741 s, 743 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.125831 s, 417 MB/s
52428800 bytes (52 MB) copied, 0.105449 s, 497 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0797467 s, 657 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0921288 s, 569 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0704724 s, 744 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0855995 s, 612 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.060466 s, 867 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0789724 s, 664 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0769451 s, 681 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0708087 s, 740 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0571781 s, 917 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0597826 s, 877 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.08733 s, 600 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0373813 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0428391 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0532008 s, 985 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0256236 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0735422 s, 713 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0277541 s, 1.9 GB/s
, 0.0591094 s, 887 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336625 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0879206 s, 596 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0529489 s, 990 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0734919 s, 713 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0458289 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0934284 s, 561 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0763844 s, 686 MB/s
52428800 bytes (52 MB) copied, 0.117114 s, 448 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0982621 s, 534 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0560819 s, 935 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.077804 s, 674 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0292171 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0578441 s, 906 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0318855 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0700546 s, 748 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468434 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0459371 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0346469 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0493155 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0606637 s, 864 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0425423 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.041362 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0411031 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0324415 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0595268 s, 881 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0320517 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0260821 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0391146 s, 1.3 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.029888 s, 1.8 GB/s
, 0.0590104 s, 888 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0568801 s, 922 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0561375 s, 934 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287567 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0369432 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0369871 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0725636 s, 723 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0778527 s, 673 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306155 s, 1.7 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0592318 s, 885 MB/s
52428800 bytes (52 MB) copied, 0.0520225 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0422555 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0405538 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0203268 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0364944 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.071854 s, 730 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0264615 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0406097 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0401906 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0473894 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0380999 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0526117 s, 997 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0515343 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0265839 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0430185 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0444626 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.052241 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287207 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0389006 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0307016 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0567709 s, 924 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0515533 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0591115 s, 887 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0484514 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0394822 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0761717 s, 688 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0743712 s, 705 MB/s
52428800 bytes (52 MB) copied, 0.0587768 s, 892 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.165286 s, 317 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.032096 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.129275 s, 406 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.118396 s, 443 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0800761 s, 655 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.181415 s, 289 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0608157 s, 862 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119183 s, 440 MB/s
52428800 bytes (52 MB) copied, 0.118552 s, 442 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0559984 s, 936 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0798619 s, 656 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0536471 s, 977 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.036476 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0513396 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0206555 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.086366 s, 607 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.042984 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.05482 s, 956 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0536847 s, 977 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0312002 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0543422 s, 965 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0477959 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0332074 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0487729 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0519834 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0661016 s, 793 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0540354 s, 970 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0279674 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0573084 s, 915 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0315179 s, 1.7 GB/s
, 0.0653953 s, 802 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0476013 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.052069 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0253611 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0234818 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0484689 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0293291 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0343981 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0387152 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0243777 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0616721 s, 850 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0177998 s, 2.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0655206 s, 800 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0548468 s, 956 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0461082 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0544532 s, 963 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0600168 s, 874 MB/s
, 0.0463483 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0619095 s, 847 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0453249 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0474826 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0259011 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0624229 s, 840 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0285891 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0476452 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0492904 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.029363 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0309157 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0456886 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0410186 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.050646 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0319609 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0356376 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.034028 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0339145 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0629364 s, 833 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0502073 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0489794 s, 1.1 GB/s
, 0.0188857 s, 2.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0402065 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0289481 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0309945 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.033026 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0291426 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308228 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0459415 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0324109 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0181824 s, 2.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0227002 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0909531 s, 576 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0530158 s, 989 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0577976 s, 907 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0208722 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0854631 s, 613 MB/s
52428800 bytes (52 MB) copied, 0.0798019 s, 657 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0220927 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102987 s, 509 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.070192 s, 747 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0180208 s, 2.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0280688 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.019248 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0277266 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0280022 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0219706 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0250413 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0345333 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0643755 s, 814 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0191699 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0638627 s, 821 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0659437 s, 795 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0366381 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0232623 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.146532 s, 358 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104847 s, 500 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0720983 s, 727 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.129921 s, 404 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0518726 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138932 s, 377 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0203443 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0373505 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0671428 s, 781 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.070003 s, 749 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.052896 s, 991 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0443658 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0817646 s, 641 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0651321 s, 805 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0322284 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0750442 s, 699 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.055083 s, 952 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522106 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.072083 s, 727 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0880546 s, 595 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0644963 s, 813 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0930097 s, 564 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382415 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.148012 s, 354 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.134399 s, 390 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0815454 s, 643 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0381421 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0971188 s, 540 MB/s
52428800 bytes (52 MB) copied, 0.0518315 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0747369 s, 702 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0396729 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0292756 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0427986 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0371662 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0607617 s, 863 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0515004 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0625556 s, 838 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0922868 s, 568 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0776179 s, 675 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0209944 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0587917 s, 892 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0666064 s, 787 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0721839 s, 726 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0428259 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0534808 s, 980 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0606534 s, 864 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0509506 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0450424 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0586787 s, 893 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0870264 s, 602 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.073903 s, 709 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0591595 s, 886 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0391465 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0403061 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.034316 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0633194 s, 828 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308991 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0551904 s, 950 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0642369 s, 816 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0811475 s, 646 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0338258 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0315001 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0455082 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382303 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0975339 s, 538 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0501044 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0421449 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0182341 s, 2.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0899584 s, 583 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.118819 s, 441 MB/s
52428800 bytes (52 MB) copied, 0.0956648 s, 548 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0692442 s, 757 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0955311 s, 549 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0518103 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0773296 s, 678 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0616493 s, 850 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0635775 s, 825 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0345571 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0185741 s, 2.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0863833 s, 607 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0751253 s, 698 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.046868 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0669889 s, 783 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.048706 s, 1.1 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0785049 s, 668 MB/s
52428800 bytes (52 MB) copied, 0.0618896 s, 847 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0829759 s, 632 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.050224 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.073933 s, 709 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468353 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0262593 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0465268 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0743839 s, 705 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0761148 s, 689 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0654428 s, 801 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0908347 s, 577 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0294961 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0415723 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.02201 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0384598 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0319825 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0251904 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0174567 s, 3.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0641068 s, 818 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0784392 s, 668 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0263487 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0196872 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0697965 s, 751 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0542202 s, 967 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0490946 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0363064 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0325019 s, 1.6 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0288743 s, 1.8 GB/s
52428800 bytes (52 MB) copied, 0.031041 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0481848 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0628435 s, 834 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0608326 s, 862 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0364321 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.053787 s, 975 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.051811 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0483918 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0365789 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0527818 s, 993 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.055063 s, 952 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0994651 s, 527 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0505106 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0186288 s, 2.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121504 s, 431 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0198447 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0561166 s, 934 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0510688 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376291 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0502482 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0776503 s, 675 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0429144 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0506707 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0700948 s, 748 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0479643 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0328911 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0540566 s, 970 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0716462 s, 732 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0714676 s, 734 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0322336 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0922967 s, 568 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0427134 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0219952 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0896242 s, 585 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0539824 s, 971 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0385642 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0396564 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0279204 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0327185 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0615017 s, 852 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0327225 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.034109 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0626723 s, 837 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0196479 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0587074 s, 893 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0511055 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0244923 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0277838 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0490597 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0358213 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0339864 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0393115 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522946 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0439376 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0645379 s, 812 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0612732 s, 856 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0624278 s, 840 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0693441 s, 756 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101359 s, 517 MB/s
52428800 bytes (52 MB) copied, 0.0629662 s, 833 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0910526 s, 576 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0845134 s, 620 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0581638 s, 901 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0402163 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0462237 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0399555 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0578572 s, 906 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0238662 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0314042 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0799568 s, 656 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0696678 s, 753 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0563852 s, 930 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0572648 s, 916 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469705 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0225214 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0389019 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0446548 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0417648 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352806 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0484883 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0354789 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0649372 s, 807 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468231 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0188637 s, 2.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0609195 s, 861 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0313234 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0372705 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0533337 s, 983 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0593155 s, 884 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0431047 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0822527 s, 637 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0436387 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0748095 s, 701 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0892973 s, 587 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0462131 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0576628 s, 909 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0353801 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0443284 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0461237 s, 1.1 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0298473 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0630591 s, 831 MB/s
52428800 bytes (52 MB) copied, 0.100182 s, 523 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0734621 s, 714 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375121 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0873714 s, 600 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0684757 s, 766 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0465599 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0726085 s, 722 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0263261 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.044607 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0550594 s, 952 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0575403 s, 911 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0509267 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0385191 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0368741 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0297486 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0263035 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109458 s, 479 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0471561 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102933 s, 509 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124884 s, 420 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0474744 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.123908 s, 423 MB/s
52428800 bytes (52 MB) copied, 0.0719243 s, 729 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0626247 s, 837 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153108 s, 342 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.205387 s, 255 MB/s
, 0.158357 s, 331 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0527317 s, 994 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0500642 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0955596 s, 549 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108408 s, 484 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.114675 s, 457 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0709892 s, 739 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0592504 s, 885 MB/s
52428800 bytes (52 MB) copied, 0.0476173 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0724144 s, 724 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0815138 s, 643 MB/s
, 0.049761 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0574939 s, 912 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0543599 s, 964 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0533984 s, 982 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0440827 s, 1.2 GB/s
, 0.0472262 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.059808 s, 877 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0508019 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352197 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.051549 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0334588 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0456078 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0399575 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0634619 s, 826 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0300071 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306153 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0395028 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376859 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0213478 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0543172 s, 965 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0278265 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469539 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0431243 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0341636 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488409 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0354886 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0198027 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.064385 s, 814 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0510534 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0478661 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0288267 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375768 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0451123 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.051501 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0567582 s, 924 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0399686 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0769667 s, 681 MB/s
, 0.062811 s, 835 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0238095 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0620105 s, 845 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0408533 s, 1.3 GB/s
52428800 bytes (52 MB) copied, 0.042949 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0199592 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0715031 s, 733 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0304695 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0304301 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0595952 s, 880 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0281882 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0569664 s, 920 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375381 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375339 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.027408 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469675 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0425325 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0408747 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0555478 s, 944 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0645323 s, 812 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0331591 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0389878 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0319158 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0304927 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0349149 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0297449 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0300634 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.023574 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0195508 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0491855 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0466363 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0549011 s, 955 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0427687 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0461995 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0355882 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336072 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.051191 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0344606 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0529602 s, 990 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0301941 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0413423 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0550999 s, 952 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0406076 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0465045 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0365584 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0354765 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0858444 s, 611 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0207028 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.068861 s, 761 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0247135 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.049251 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0296917 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0656181 s, 799 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287402 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0860181 s, 610 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0600371 s, 873 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0454347 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0460529 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0328309 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0480234 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0357174 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0734875 s, 713 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0386737 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0849547 s, 617 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0773248 s, 678 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0454963 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0889558 s, 589 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0568421 s, 922 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336394 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0702403 s, 746 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0504706 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0337381 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0425364 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.03268 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0389385 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0442037 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0451065 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0419907 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0520169 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0363501 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.081363 s, 644 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0571258 s, 918 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0535585 s, 979 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0793549 s, 661 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0243355 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0764663 s, 686 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0305472 s, 1.7 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0487622 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.0653373 s, 802 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.068221 s, 769 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0367555 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287144 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0938769 s, 558 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0679238 s, 772 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0303394 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0367265 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.057731 s, 908 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0439954 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0403096 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.03582 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0384554 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.05806 s, 903 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0235798 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0307781 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0485592 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.037159 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0462745 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0343217 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0383154 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0549502 s, 954 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0321653 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0470152 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0868512 s, 604 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0623294 s, 841 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0977885 s, 536 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.028057 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0299917 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0326909 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.036972 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0296344 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0315657 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0324897 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0392776 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0294071 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0270686 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0305923 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0350947 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0531981 s, 986 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0393625 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0694808 s, 755 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0685391 s, 765 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0296096 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0639134 s, 820 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0513733 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0325267 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336129 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352006 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0286096 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0683799 s, 767 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0499002 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0369465 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0349961 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0422789 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0349683 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0491885 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0724235 s, 724 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0560404 s, 936 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0499398 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0642743 s, 816 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.093879 s, 558 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0780622 s, 672 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0670581 s, 782 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0425932 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0539867 s, 971 MB/s
52428800 bytes (52 MB) copied, 0.0875622 s, 599 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0737503 s, 711 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0330748 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0544967 s, 962 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0366432 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0719384 s, 729 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488113 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0620707 s, 845 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0254914 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0557871 s, 940 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0395464 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0280659 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0413513 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0316964 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0284654 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0378907 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0297728 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0554214 s, 946 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.044685 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100386 s, 522 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0711636 s, 737 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0703596 s, 745 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0895063 s, 586 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0581908 s, 901 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0769334 s, 681 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.129888 s, 404 MB/s
52428800 bytes (52 MB) copied, 0.0488311 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.054461 s, 963 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0776758 s, 675 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0475469 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.078871 s, 665 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0392991 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0519768 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.040861 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.048697 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360941 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0579191 s, 905 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0209979 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0551174 s, 951 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109824 s, 477 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0390201 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0636924 s, 823 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0582664 s, 900 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0922575 s, 568 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0855889 s, 613 MB/s
52428800 bytes (52 MB) copied, 0.0661098 s, 793 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0276506 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0293647 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0628216 s, 835 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0219134 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0612918 s, 855 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0659076 s, 795 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0619548 s, 846 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.047831 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0225248 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0430048 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0447313 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0492999 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.046552 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0266341 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0737621 s, 711 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0459512 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.0543999 s, 964 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104677 s, 501 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13079 s, 401 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.073482 s, 713 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12187 s, 430 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.137962 s, 380 MB/s
, 0.0731549 s, 717 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0616039 s, 851 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0951096 s, 551 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0433829 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0585072 s, 896 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0265241 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0458085 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469135 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.089341 s, 587 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0920092 s, 570 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0621844 s, 843 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109719 s, 478 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0914953 s, 573 MB/s
52428800 bytes (52 MB) copied, 0.138019 s, 380 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.175236 s, 299 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0635773 s, 825 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0821176 s, 638 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.21235 s, 247 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.246326 s, 213 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.285057 s, 184 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.177798 s, 295 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522359 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0585259 s, 896 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0980733 s, 535 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306198 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107598 s, 487 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0369866 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0545122 s, 962 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0471127 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.066008 s, 794 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0559595 s, 937 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0654033 s, 802 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0501194 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0738609 s, 710 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0718134 s, 730 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0849727 s, 617 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0674728 s, 777 MB/s
52428800 bytes (52 MB) copied, 0.060483 s, 867 MB/s
52428800 bytes (52 MB) copied, 0.0574085 s, 913 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.239724 s, 219 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.204479 s, 256 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.242804 s, 216 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.26059 s, 201 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0804832 s, 651 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0486062 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.211149 s, 248 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0914354 s, 573 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.132428 s, 396 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0201949 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0878878 s, 597 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0927672 s, 565 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.053088 s, 988 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.044008 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0499156 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0414388 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0700651 s, 748 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0401771 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.026724 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.062602 s, 837 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0777736 s, 674 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0584035 s, 898 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.060748 s, 863 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0769659 s, 681 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0764116 s, 686 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0246192 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0393743 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0525632 s, 997 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0300844 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0328543 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0434855 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0404996 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0843197 s, 622 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0321501 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0922706 s, 568 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0719617 s, 729 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0982186 s, 534 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0552916 s, 948 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0641887 s, 817 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0772184 s, 679 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0289315 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0723117 s, 725 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0566813 s, 925 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0600556 s, 873 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0386165 s, 1.4 GB/s
52428800 bytes (52 MB) copied, 0.0443512 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0343005 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0753269 s, 696 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.089078 s, 589 MB/s
52428800 bytes (52 MB) copied, 0.0798821 s, 656 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0377833 s, 1.4 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.026845 s, 2.0 GB/s
52428800 bytes (52 MB) copied, 0.0652773 s, 803 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0395758 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0266697 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0390778 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0450575 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0466739 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0571851 s, 917 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469731 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0608922 s, 861 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0815712 s, 643 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0956948 s, 548 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12525 s, 419 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103221 s, 508 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0701889 s, 747 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100755 s, 520 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0754194 s, 695 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0748023 s, 701 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0750957 s, 698 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.062681 s, 836 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0367788 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0358263 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.047118 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0632062 s, 829 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0275085 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0473079 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370995 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0803623 s, 652 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0801736 s, 654 MB/s
52428800 bytes (52 MB) copied, 0.0771221 s, 680 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287938 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0578175 s, 907 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0410729 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0812377 s, 645 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0666798 s, 786 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0416753 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0516051 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0649591 s, 807 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0210379 s, 2.5 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0719183 s, 729 MB/s
52428800 bytes (52 MB) copied, 0.0590608 s, 888 MB/s
52428800 bytes (52 MB) copied, 0.0388715 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.072299 s, 725 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0849751 s, 617 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0711143 s, 737 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0458395 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0557424 s, 941 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0567578 s, 924 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0524135 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0434142 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0335178 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0294401 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0764478 s, 686 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0984585 s, 532 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0239661 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0947632 s, 553 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0344524 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0369188 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0545684 s, 961 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0628464 s, 834 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0565529 s, 927 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0480006 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0461551 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352294 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.032627 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0466298 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0415996 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0381388 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0927496 s, 565 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0579215 s, 905 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0347676 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360674 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376352 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0380144 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287198 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336997 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0632134 s, 829 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0629397 s, 833 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0470095 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0326388 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0353883 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.029442 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0282319 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468348 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0596113 s, 880 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0693407 s, 756 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0226627 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0516846 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.039698 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0736632 s, 712 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0330319 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0393646 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0523471 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.045759 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0624668 s, 839 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0576133 s, 910 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.037485 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0437836 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0583271 s, 899 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0687101 s, 763 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0591948 s, 886 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0553566 s, 947 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0367426 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382981 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0325507 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0763388 s, 687 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0291077 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0487086 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0867631 s, 604 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0638966 s, 821 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0273909 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0935505 s, 560 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0300314 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0504846 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0671649 s, 781 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0326459 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0445577 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0415391 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0355492 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0697865 s, 751 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0421959 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336372 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0696788 s, 752 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.093575 s, 560 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0900247 s, 582 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0977563 s, 536 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0284767 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0553562 s, 947 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0425169 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0615969 s, 851 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.054844 s, 956 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0217956 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.144126 s, 364 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0454714 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.060808 s, 862 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.043026 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0413128 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0636662 s, 823 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110279 s, 475 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0378711 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0392067 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0479618 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0528822 s, 991 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.037644 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0454596 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0305352 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0228239 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0562919 s, 931 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0759692 s, 690 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0472428 s, 1.1 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153677 s, 341 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101111 s, 519 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0869199 s, 603 MB/s
52428800 bytes (52 MB) copied, 0.0837026 s, 626 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11771 s, 445 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0694192 s, 755 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0549889 s, 953 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522643 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0537646 s, 975 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0848038 s, 618 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0956016 s, 548 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0569361 s, 921 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0873612 s, 600 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0739152 s, 709 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0768671 s, 682 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0530628 s, 988 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360844 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0496232 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0481043 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0532569 s, 984 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0495806 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0367068 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0484706 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0290204 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.035338 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0582817 s, 900 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0764021 s, 686 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0363088 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0839594 s, 624 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0383031 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.052386 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112081 s, 468 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127483 s, 411 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0247744 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.065526 s, 800 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0555357 s, 944 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0419452 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0402373 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0600945 s, 872 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0587135 s, 893 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0541006 s, 969 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.026684 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0726249 s, 722 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0699452 s, 750 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0478038 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.078744 s, 666 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0661279 s, 793 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0244071 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0371886 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0597692 s, 877 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0592621 s, 885 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0476581 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0811669 s, 646 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0330366 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.133894 s, 392 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0618435 s, 848 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0750483 s, 699 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0612516 s, 856 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0702875 s, 746 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0385394 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115801 s, 453 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0841252 s, 623 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106476 s, 492 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.071631 s, 732 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0814682 s, 644 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0261437 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101869 s, 515 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0414988 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0450771 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0497829 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0599478 s, 875 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.026096 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0452352 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0552173 s, 949 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0676837 s, 775 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0941025 s, 557 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0525415 s, 998 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0630844 s, 831 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0309773 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0348934 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0272163 s, 1.9 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.079796 s, 657 MB/s
, 0.0615725 s, 851 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0538681 s, 973 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0616409 s, 851 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.024604 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0678749 s, 772 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112026 s, 468 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0350454 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.037711 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0509972 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0313236 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0585289 s, 896 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0542654 s, 966 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0378138 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469474 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0513753 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0463833 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0435855 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0405068 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0464 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0472701 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0452532 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0281625 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.048812 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0338559 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0265792 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.030444 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0512436 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0613505 s, 855 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0689122 s, 761 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0335083 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0455845 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287929 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0581605 s, 901 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382796 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.088475 s, 593 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0526924 s, 995 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0858369 s, 611 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0519378 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.023951 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0730794 s, 717 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.036573 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0438989 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0364091 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0367116 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.046526 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0508495 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0219121 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0363911 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0294492 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0479212 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0411796 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0433186 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0892301 s, 588 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0718422 s, 730 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100125 s, 524 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104864 s, 500 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0778416 s, 674 MB/s
, 0.125021 s, 419 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0801933 s, 654 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0586045 s, 895 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308742 s, 1.7 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138097 s, 380 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.100514 s, 522 MB/s
, 0.0785599 s, 667 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0592277 s, 885 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0327264 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0342465 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0422293 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0325729 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0518393 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0379056 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0651194 s, 805 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0400567 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0750209 s, 699 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311089 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0344265 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0429232 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0367894 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0283226 s, 1.9 GB/s
, 0.0232662 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0242048 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287971 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.025831 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0584217 s, 897 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0627318 s, 836 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0639233 s, 820 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0289976 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0342515 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0348465 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.046619 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0353324 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0359048 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.031518 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0451211 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375674 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0279638 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258269 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0456158 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0365414 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360363 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0434808 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0386856 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0326234 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311388 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0400544 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0473805 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0317219 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0339995 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0467017 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0552127 s, 950 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0304532 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0720255 s, 728 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0327351 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0415127 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0225112 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0256814 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0364963 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0267397 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0317109 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0435976 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0430744 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0586596 s, 894 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360882 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0460887 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0639222 s, 820 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0387348 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0229702 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0267112 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0307303 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0377791 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0255836 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0296169 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0332751 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0428593 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0480778 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0257841 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0216436 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0312788 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0241722 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0354681 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0390513 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.065119 s, 805 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0279768 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0324525 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0317162 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0494798 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0483241 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0538315 s, 974 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0427444 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0363481 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0323258 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.062638 s, 837 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0591675 s, 886 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0775759 s, 676 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.073933 s, 709 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0498227 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.0543025 s, 965 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0687266 s, 763 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.070548 s, 743 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0715798 s, 732 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.041025 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0508862 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.050315 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0223679 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0232691 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0228381 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0540723 s, 970 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0511648 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382681 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0293364 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0456417 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.03888 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0347053 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0203509 s, 2.6 GB/s
, 0.0437817 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0342323 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0454212 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0506558 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0373313 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0506755 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0392294 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0446871 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.038279 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0477645 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0482427 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0420512 s, 1.2 GB/s
, 0.0339274 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0330565 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0278004 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.02658 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0328635 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.043838 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0404194 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0472032 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0441703 s, 1.2 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0553962 s, 946 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0556517 s, 942 MB/s
52428800 bytes (52 MB) copied, 0.0754242 s, 695 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0592654 s, 885 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10029 s, 523 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0510491 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.063714 s, 823 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0958194 s, 547 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0344703 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0440484 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0550967 s, 952 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0341108 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0446751 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0540691 s, 970 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0698653 s, 750 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0286438 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0775739 s, 676 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0333314 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0750547 s, 699 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0517508 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0643882 s, 814 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0613402 s, 855 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0585915 s, 895 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0368917 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311753 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0447913 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0380696 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0357988 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0399205 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0343942 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.037503 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0600963 s, 872 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0313259 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0393409 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0310364 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0309441 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.032179 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0359474 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0412819 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0385411 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0270274 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0319923 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0397109 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0318974 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0604363 s, 868 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0828201 s, 633 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0756154 s, 693 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.044235 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0335752 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0466591 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.0938975 s, 558 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382707 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375529 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0558227 s, 939 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0301225 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0309476 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0329113 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0398289 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.05026 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0331773 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0550149 s, 953 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0331596 s, 1.6 GB/s
, 0.0323481 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0310165 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0298191 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.020045 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0505388 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382012 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488462 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0727632 s, 721 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0340406 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0615111 s, 852 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.047561 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0443579 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0413084 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0292703 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370097 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0238806 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0517886 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0539539 s, 972 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0325693 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0685556 s, 765 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0355292 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0397147 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0388024 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.029412 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0301209 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0601026 s, 872 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0223783 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308877 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0247672 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0387263 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0371962 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0410154 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0409011 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0475721 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0489796 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0254087 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0556708 s, 942 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0547216 s, 958 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0304522 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0621353 s, 844 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0316646 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0462626 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0651043 s, 805 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0249043 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0492985 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100319 s, 523 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0951961 s, 551 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0864977 s, 606 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0235912 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0283613 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0485117 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0309953 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0435363 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0300891 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0346329 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0430033 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0235175 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0281182 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0303975 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0441481 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0318355 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0524567 s, 999 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0557303 s, 941 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0530993 s, 987 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0890558 s, 589 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0851638 s, 616 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0974524 s, 538 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0580843 s, 903 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0950281 s, 552 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0396663 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0661256 s, 793 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12552 s, 418 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488004 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0642082 s, 817 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0372184 s, 1.4 GB/s
, 0.0627201 s, 836 MB/s
52428800 bytes (52 MB) copied, 0.064971 s, 807 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0320191 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0335221 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0323669 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0313071 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0574342 s, 913 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0553773 s, 947 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0423767 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0557335 s, 941 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0364623 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0580572 s, 903 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.063597 s, 824 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0374332 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0656 s, 799 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0629931 s, 832 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0389804 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308197 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.044219 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0577182 s, 908 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0304172 s, 1.7 GB/s
, 0.0303276 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0476046 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0363855 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0592086 s, 885 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0320131 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0549896 s, 953 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468476 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0537315 s, 976 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.047312 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0560481 s, 935 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0600112 s, 874 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0452241 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0601721 s, 871 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0482928 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.0317449 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0261439 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.053986 s, 971 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0626902 s, 836 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311683 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0727524 s, 721 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0534309 s, 981 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0744497 s, 704 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0483082 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0397311 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0487241 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0447439 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0536161 s, 978 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0426468 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0406615 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0401132 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0464755 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0407638 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0271203 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0402984 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0887422 s, 591 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0754607 s, 695 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0908186 s, 577 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0514491 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0944423 s, 555 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.196516 s, 267 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0871112 s, 602 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.216878 s, 242 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.201512 s, 260 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.152049 s, 345 MB/s
52428800 bytes (52 MB) copied, 0.14127 s, 371 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.16947 s, 309 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.154992 s, 338 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.22343 s, 235 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.284868 s, 184 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131613 s, 398 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0628394 s, 834 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0743754 s, 705 MB/s
52428800 bytes (52 MB) copied, 0.111794 s, 469 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0622303 s, 842 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128702 s, 407 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376716 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136085 s, 385 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0327447 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0898156 s, 584 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0492947 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258409 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0322808 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.051323 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0402466 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0341977 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0491451 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0460419 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0730543 s, 718 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382598 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0699126 s, 750 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0565123 s, 928 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105274 s, 498 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0832812 s, 630 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0353655 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0613636 s, 854 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0636252 s, 824 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.043858 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0353539 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0272393 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0444499 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0397323 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0413917 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0487509 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0662563 s, 791 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0363553 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0567097 s, 925 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0524873 s, 999 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0849741 s, 617 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0512001 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0651547 s, 805 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0807512 s, 649 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488863 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0254186 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112175 s, 467 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.052034 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.046327 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0200645 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0354383 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0499765 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0659546 s, 795 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.049838 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0499382 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.083357 s, 629 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121188 s, 433 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0526637 s, 996 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0763956 s, 686 MB/s
52428800 bytes (52 MB) copied, 0.0721805 s, 726 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488304 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0683787 s, 767 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0450381 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0228089 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0254019 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12994 s, 403 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0968224 s, 541 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0642563 s, 816 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0432968 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375669 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0416239 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360061 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122909 s, 427 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0968446 s, 541 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0732029 s, 716 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0716285 s, 732 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11979 s, 438 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0843631 s, 621 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.162138 s, 323 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0851623 s, 616 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0818614 s, 640 MB/s
52428800 bytes (52 MB) copied, 0.0591764 s, 886 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0921766 s, 569 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0570055 s, 920 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0785269 s, 668 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0959122 s, 547 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.093786 s, 559 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0876391 s, 598 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0853492 s, 614 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0407539 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0463337 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0656384 s, 799 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0806775 s, 650 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0380893 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11935 s, 439 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.02017 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0436092 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0574652 s, 912 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0651606 s, 805 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0556382 s, 942 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0530412 s, 988 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0537331 s, 976 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0526199 s, 996 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0374181 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0419058 s, 1.3 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0362681 s, 1.4 GB/s
52428800 bytes (52 MB) copied, 0.0757123 s, 692 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0495284 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0458403 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0264177 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0313554 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375493 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0297281 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0391398 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0867785 s, 604 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0339025 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0812257 s, 645 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0797622 s, 657 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0283377 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0417446 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0730591 s, 718 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0203392 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0368684 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0232511 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0470479 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488293 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0552297 s, 949 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.084018 s, 624 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0860138 s, 610 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.129164 s, 406 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112998 s, 464 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128174 s, 409 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0720605 s, 728 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0582783 s, 900 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0485095 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106108 s, 494 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0503104 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0388329 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0243749 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0178919 s, 2.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0757504 s, 692 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106579 s, 492 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0943459 s, 556 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0794431 s, 660 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0885228 s, 592 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0262284 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0616532 s, 850 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0447772 s, 1.2 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.173107 s, 303 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.198667 s, 264 MB/s
52428800 bytes (52 MB) copied, 0.0856688 s, 612 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104998 s, 499 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.159692 s, 328 MB/s
52428800 bytes (52 MB) copied, 0.112851 s, 465 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.05022 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106061 s, 494 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0739947 s, 709 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0338982 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0409043 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0482 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0715579 s, 733 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0702472 s, 746 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102539 s, 511 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116824 s, 449 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128941 s, 407 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0629092 s, 833 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103995 s, 504 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120661 s, 435 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102341 s, 512 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.141216 s, 371 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0607941 s, 862 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.034589 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105286 s, 498 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.218827 s, 240 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.201712 s, 260 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.174774 s, 300 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.191177 s, 274 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0838945 s, 625 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.156901 s, 334 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0538802 s, 973 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106669 s, 492 MB/s
, 0.0926058 s, 566 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0648316 s, 809 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.125397 s, 418 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128331 s, 409 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.143164 s, 366 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0600724 s, 873 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0745083 s, 704 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128451 s, 408 MB/s
52428800 bytes (52 MB) copied, 0.0914468 s, 573 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.140688 s, 373 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0891338 s, 588 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101122 s, 518 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0904113 s, 580 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0766919 s, 684 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0877663 s, 597 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0742326 s, 706 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.077046 s, 680 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.035825 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0584758 s, 897 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0610306 s, 859 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0389808 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0420727 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0761791 s, 688 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0509679 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0952612 s, 550 MB/s
52428800 bytes (52 MB) copied, 0.0433687 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0495024 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0249851 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0704279 s, 744 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0486904 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0642491 s, 816 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0769195 s, 682 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0982086 s, 534 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0855218 s, 613 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112509 s, 466 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0538692 s, 973 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0671923 s, 780 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0581891 s, 901 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0400468 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0703362 s, 745 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0507227 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0411334 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0601279 s, 872 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0555055 s, 945 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0816087 s, 642 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11742 s, 447 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.085376 s, 614 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120131 s, 436 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0824312 s, 636 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0728101 s, 720 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.132036 s, 397 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0339722 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.252427 s, 208 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136663 s, 384 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.085251 s, 615 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.169351 s, 310 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.053591 s, 978 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101095 s, 519 MB/s
52428800 bytes (52 MB) copied, 0.184152 s, 285 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.192457 s, 272 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0697155 s, 752 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128282 s, 409 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469942 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0416129 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0528497 s, 992 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0592658 s, 885 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0435184 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0457895 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.051998 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0542796 s, 966 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.14201 s, 369 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0416306 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.062081 s, 845 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.09023 s, 581 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121281 s, 432 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107916 s, 486 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.172744 s, 304 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103362 s, 507 MB/s
, 0.0907573 s, 578 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.23367 s, 224 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.114155 s, 459 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0527092 s, 995 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0480078 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0725148 s, 723 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111633 s, 470 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.049473 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0421647 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116306 s, 451 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488741 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0552276 s, 949 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0394362 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.044742 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0314322 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0363971 s, 1.4 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0397499 s, 1.3 GB/s
, 0.0397706 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0257622 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0296308 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0639995 s, 819 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0614944 s, 853 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0416675 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0283041 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469959 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0366057 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370719 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0347016 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0440306 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0579339 s, 905 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0630096 s, 832 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0709545 s, 739 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0960582 s, 546 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0820369 s, 639 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0800548 s, 655 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0747706 s, 701 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0333491 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0312961 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0514842 s, 1.0 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0509451 s, 1.0 GB/s
52428800 bytes (52 MB) copied, 0.021506 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0272042 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0362106 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0533095 s, 983 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0713792 s, 735 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0595647 s, 880 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101177 s, 518 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376029 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0735188 s, 713 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0610591 s, 859 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0716421 s, 732 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0800655 s, 655 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0507199 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0993109 s, 528 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0365888 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0622591 s, 842 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109579 s, 478 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0696857 s, 752 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468054 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370564 s, 1.4 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0189531 s, 2.8 GB/s
52428800 bytes (52 MB) copied, 0.134317 s, 390 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119801 s, 438 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0701308 s, 748 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0909904 s, 576 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0231772 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0597089 s, 878 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0732955 s, 715 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0886507 s, 591 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.055763 s, 940 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0686552 s, 764 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0632023 s, 830 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0566055 s, 926 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0505906 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0510173 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110995 s, 472 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0342748 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0318672 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0513303 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0413055 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0342876 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0390477 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0286288 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.037647 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0298741 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0338863 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0563203 s, 931 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0351116 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0358913 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.028941 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0385783 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0498163 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0639912 s, 819 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.073139 s, 717 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0277515 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0800081 s, 655 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0704794 s, 744 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13347 s, 393 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127464 s, 411 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.036929 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0580988 s, 902 MB/s
52428800 bytes (52 MB) copied, 0.167459 s, 313 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0691053 s, 759 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0467706 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0839421 s, 625 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0604955 s, 867 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0379144 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0447655 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0662681 s, 791 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0644483 s, 814 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0501464 s, 1.0 GB/s
52428800 bytes (52 MB) copied, 0.0567602 s, 924 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0441851 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287495 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375108 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0396428 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0570377 s, 919 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0394257 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287049 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0348552 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0424265 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0412113 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0431955 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0388083 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0316272 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0415408 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0483588 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0229636 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376643 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308807 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0315673 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0349845 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0351948 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0259914 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0751428 s, 698 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0778443 s, 674 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0452968 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0542785 s, 966 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0643347 s, 815 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.070854 s, 740 MB/s
52428800 bytes (52 MB) copied, 0.0767767 s, 683 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0759229 s, 691 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0381572 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0451882 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0770389 s, 681 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0387631 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0354417 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0687667 s, 762 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0550241 s, 953 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.05071 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0525724 s, 997 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0681218 s, 770 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0558647 s, 938 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0315305 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0676512 s, 775 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258655 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0620617 s, 845 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0661179 s, 793 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0711992 s, 736 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0309051 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0296572 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0433053 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0331191 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0415093 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0881375 s, 595 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0499037 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0403796 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0351992 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0293391 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0626102 s, 837 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0449373 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0526749 s, 995 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0476828 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0211943 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0289091 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0386157 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0565621 s, 927 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.033603 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0332258 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0409256 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0711436 s, 737 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0707988 s, 741 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0955534 s, 549 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0529469 s, 990 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0556617 s, 942 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0772621 s, 679 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0470756 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0774459 s, 677 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0935023 s, 561 MB/s
52428800 bytes (52 MB) copied, 0.0879247 s, 596 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0448607 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0966644 s, 542 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12749 s, 411 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0832622 s, 630 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0400979 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0435337 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0571061 s, 918 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0202611 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0420568 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311993 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0504898 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0331464 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0678795 s, 772 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0793945 s, 660 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0778041 s, 674 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0856574 s, 612 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0494475 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0909405 s, 577 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107313 s, 489 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0442297 s, 1.2 GB/s
, 0.0737153 s, 711 MB/s
52428800 bytes (52 MB) copied, 0.100542 s, 521 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124776 s, 420 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.150907 s, 347 MB/s
52428800 bytes (52 MB) copied, 0.149087 s, 352 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126759 s, 414 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0398498 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.123358 s, 425 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0718955 s, 729 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121536 s, 431 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0494565 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0598739 s, 876 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0667269 s, 786 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0596413 s, 879 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0433316 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0726055 s, 722 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0540644 s, 970 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0613146 s, 855 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0562284 s, 932 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0796084 s, 659 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0331028 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0355797 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0508017 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.045541 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0509342 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0563726 s, 930 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.030202 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0355691 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0461064 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.038917 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0374128 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0461926 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0660646 s, 794 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.055959 s, 937 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0664268 s, 789 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0594816 s, 881 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0808719 s, 648 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.084952 s, 617 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108785 s, 482 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0459303 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0568278 s, 923 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0675826 s, 776 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0463727 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0608029 s, 862 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0694861 s, 755 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0487436 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.114495 s, 458 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0451003 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0637961 s, 822 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0372099 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0206466 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0419102 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0349271 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0598301 s, 876 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0986606 s, 531 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0387472 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0502017 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0424852 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0392505 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.027533 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0402074 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0279011 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0377067 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0348872 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0484961 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0496336 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488702 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469006 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.078088 s, 671 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0449574 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0764945 s, 685 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0716081 s, 732 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0434439 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0260761 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0531864 s, 986 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0717889 s, 730 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0788689 s, 665 MB/s
52428800 bytes (52 MB) copied, 0.0647767 s, 809 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117979 s, 444 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0596897 s, 878 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0847156 s, 619 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0716564 s, 732 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0543341 s, 965 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0658084 s, 797 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0192956 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0322569 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0549589 s, 954 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0365847 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.0637647 s, 822 MB/s
52428800 bytes (52 MB) copied, 0.0763229 s, 687 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0639498 s, 820 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0292563 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0534928 s, 980 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0686103 s, 764 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10103 s, 519 MB/s
52428800 bytes (52 MB) copied, 0.101917 s, 514 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.066445 s, 789 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382907 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117494 s, 446 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0939436 s, 558 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0574464 s, 913 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0536834 s, 977 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0623145 s, 841 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0586032 s, 895 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.150925 s, 347 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0345623 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0385171 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0484575 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0331811 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0514237 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0598511 s, 876 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0649272 s, 808 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0592171 s, 885 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0361459 s, 1.5 GB/s
Finish ========
/mnt/lustre/f44.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    2
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x2000059f1
      lmm_object_id:     0x6
      lmm_fid:           [0x2000059f1:0x6:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x1a4:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
llite.lustre-ffff8800a99f2000.stats=
snapshot_time             1746138750.584619759 secs.nsecs
start_time                1746138690.035468432 secs.nsecs
elapsed_time              60.549151327 secs.nsecs
read_bytes                3556 samples [bytes] 1048576 1048576 3728736256 3909863348371456
read                      132800 samples [usecs] 154 534468 166679940 6163320527338
ioctl                     1 samples [reqs]
open                      2663 samples [usecs] 5 226194 7081768 400530187246
close                     2663 samples [usecs] 5 164347 3927988 212989053480
seek                      2655 samples [usecs] 2 9413 25821 90070393
setattr                   1 samples [usecs] 23551 23551 23551 554649601
getattr                   17 samples [usecs] 116 2099 7028 8785534
mknod                     1 samples [usecs] 14924 14924 14924 222725776
inode_permission          8225 samples [usecs] 1 64102 5458201 78973380803
opencount                 2664 samples [reqs] 1 2663 3547117 6298495645
openclosetime             2662 samples [usecs] 5 213252 55909749 2536998601163
pcc_attach                1 samples [reqs]
pcc_detach                229 samples [reqs]
pcc_auto_attach           229 samples [reqs]
pcc_hit_bytes             2587 samples [bytes] 52428800 52428800 135633305600 7111091452641280000
pcc_attach_bytes          1 samples [bytes] 52428800 52428800 52428800 2748779069440000
llite.lustre-ffff8800b6371000.stats=
snapshot_time             1746138750.584752922 secs.nsecs
start_time                1746138690.035491249 secs.nsecs
elapsed_time              60.549261673 secs.nsecs
attach 1 detach 229 auto_attach 229
llite.lustre-ffff8800a99f2000.pcc_async_threshold=1073741824
llite.lustre-ffff8800b6371000.pcc_async_threshold=1073741824
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d44.sanity-pcc is idle now
PASS 44 (67s)

== sanity-pcc test 45: Concurrent read access from two mount points ========================================================== 18:32:34 (1746138754)
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 0.178381 s, 588 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d45.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:   1024/102400             done                            
Creating filesystem with 102400 1k blocks and 25688 inodes
Filesystem UUID: 99bea1b0-3f58-4c4a-985a-26c040eb0b29
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345, 73729

Allocating group tables:  0/13     done                            
Writing inode tables:  0/13     done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information:  0/13     done

/tmp/f45.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=99bea1b0-3f58-4c4a-985a-26c040eb0b29 (extents) (64bit) (large files) (huge files)
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 0.182869 s, 573 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
/tmp/f45.sanity-pcc on /mnt/pcc.d45.sanity-pcc type ext4 (rw,relatime,quota,usrquota,grpquota,data=ordered)
umount: /mnt/pcc.d45.sanity-pcc.2: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
/tmp/f45.sanity-pcc on /mnt/pcc.d45.sanity-pcc type ext4 (rw,relatime,quota,usrquota,grpquota,data=ordered)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:   1024/102400             done                            
Creating filesystem with 102400 1k blocks and 25688 inodes
Filesystem UUID: 7953c51d-4801-4714-8afe-af6d608391b9
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345, 73729

Allocating group tables:  0/13     done                            
Writing inode tables:  0/13     done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information:  0/13     done

/tmp/f45.sanity-pcc.2: Linux rev 1.0 ext4 filesystem data, UUID=7953c51d-4801-4714-8afe-af6d608391b9 (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d45.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={0}
pcc:
  -
    pccpath: /mnt/pcc.d45.sanity-pcc.2
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={0}
llite.lustre-ffff8800a99f2000.pcc_async_threshold=0
llite.lustre-ffff8800b6371000.pcc_async_threshold=0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.582112 s, 90.1 MB/s
/mnt/lustre/f45.sanity-pcc
lmm_magic:         0x0BD10BD0
lmm_seq:           0x2000059f1
lmm_object_id:     0x8
lmm_fid:           [0x2000059f1:0x8:0x0]
lmm_stripe_count:  1
lmm_stripe_size:   4194304
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 1
	obdidx		 objid		 objid		 group
	     1	           421	        0x1a5	   0x280000400

50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.688833 s, 76.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.726888 s, 72.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.72441 s, 72.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.742423 s, 70.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.686736 s, 76.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.695128 s, 75.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.688365 s, 76.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.729642 s, 71.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.774347 s, 67.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.736922 s, 71.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.746436 s, 70.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.698449 s, 75.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.705083 s, 74.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.702676 s, 74.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.772388 s, 67.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.750015 s, 69.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.894726 s, 58.6 MB/s
, 0.893558 s, 58.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.894345 s, 58.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.890554 s, 58.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.897394 s, 58.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.90195 s, 58.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.92795 s, 56.5 MB/s
, 0.897425 s, 58.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.902952 s, 58.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.904868 s, 57.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.930986 s, 56.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.909717 s, 57.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.910014 s, 57.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.907494 s, 57.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.914827 s, 57.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.915902 s, 57.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136441 s, 384 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0572744 s, 915 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0300809 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.200635 s, 261 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.252776 s, 207 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.162112 s, 323 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0697711 s, 751 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.199586 s, 263 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10267 s, 511 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0890651 s, 589 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.123661 s, 424 MB/s
, 0.107501 s, 488 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157791 s, 332 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108665 s, 482 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.159153 s, 329 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.144225 s, 364 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.166738 s, 314 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.180401 s, 291 MB/s
52428800 bytes (52 MB) copied, 0.171793 s, 305 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.132389 s, 396 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.19881 s, 264 MB/s
52428800 bytes (52 MB) copied, 0.189613 s, 277 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.186069 s, 282 MB/s
52428800 bytes (52 MB) copied, 0.212249 s, 247 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.224166 s, 234 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.216489 s, 242 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.148822 s, 352 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.222766 s, 235 MB/s
52428800 bytes (52 MB) copied, 0.125897 s, 416 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0804453 s, 652 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.251868 s, 208 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0612663 s, 856 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0971115 s, 540 MB/s
52428800 bytes (52 MB) copied, 0.0878411 s, 597 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.211289 s, 248 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0604768 s, 867 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0998204 s, 525 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101267 s, 518 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0671583 s, 781 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0205695 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.037126 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336798 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0310858 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0426543 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0435431 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0440494 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0405034 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.069167 s, 758 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0373735 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0388156 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0578647 s, 906 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0926852 s, 566 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.050966 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522412 s, 1.0 GB/s
, 0.0789077 s, 664 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136718 s, 383 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0928104 s, 565 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.073649 s, 712 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0288173 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0247288 s, 2.1 GB/s
, 0.0390344 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0410362 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0303548 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.034139 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0256288 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0217282 s, 2.4 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336103 s, 1.6 GB/s
52428800 bytes (52 MB) copied, 0.0652272 s, 804 MB/s
52428800 bytes (52 MB) copied, 0.0653059 s, 803 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.125754 s, 417 MB/s
52428800 bytes (52 MB) copied, 0.125997 s, 416 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12474 s, 420 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106321 s, 493 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.129329 s, 405 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136968 s, 383 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115931 s, 452 MB/s
52428800 bytes (52 MB) copied, 0.137479 s, 381 MB/s
52428800 bytes (52 MB) copied, 0.18572 s, 282 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116469 s, 450 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.182588 s, 287 MB/s
52428800 bytes (52 MB) copied, 0.155635 s, 337 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.140911 s, 372 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.183348 s, 286 MB/s
52428800 bytes (52 MB) copied, 0.116865 s, 449 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.154274 s, 340 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109945 s, 477 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0807438 s, 649 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.130868 s, 401 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.133262 s, 393 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149635 s, 350 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0580224 s, 904 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128143 s, 409 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0715207 s, 733 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0865636 s, 606 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0278562 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0705033 s, 744 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0508705 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0440341 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0407364 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336186 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0235557 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0259098 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287579 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0290581 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0227749 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0231234 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370568 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0422715 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0417213 s, 1.3 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.025279 s, 2.1 GB/s
, 0.0367509 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0255216 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0322741 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0555034 s, 945 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0692073 s, 758 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0936346 s, 560 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0762548 s, 688 MB/s
, 0.124444 s, 421 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108037 s, 485 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12368 s, 424 MB/s
52428800 bytes (52 MB) copied, 0.0910943 s, 576 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128988 s, 406 MB/s
52428800 bytes (52 MB) copied, 0.142225 s, 369 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.125947 s, 416 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13041 s, 402 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115097 s, 456 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.105426 s, 497 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11666 s, 449 MB/s
52428800 bytes (52 MB) copied, 0.127615 s, 411 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112056 s, 468 MB/s
52428800 bytes (52 MB) copied, 0.102195 s, 513 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.133419 s, 393 MB/s
52428800 bytes (52 MB) copied, 0.161627 s, 324 MB/s
52428800 bytes (52 MB) copied, 0.102359 s, 512 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.182282 s, 288 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.13134 s, 399 MB/s
, 0.104748 s, 501 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.156114 s, 336 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.19003 s, 276 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104674 s, 501 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0854212 s, 614 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0513415 s, 1.0 GB/s
52428800 bytes (52 MB) copied, 0.047339 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0277196 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0340353 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0264508 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0390037 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0219429 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0768655 s, 682 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0641284 s, 818 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0693707 s, 756 MB/s
52428800 bytes (52 MB) copied, 0.0896033 s, 585 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0978883 s, 536 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0882504 s, 594 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0369237 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0377076 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0932416 s, 562 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0583482 s, 899 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0566234 s, 926 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0605856 s, 865 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0292235 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0426589 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.040313 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0505195 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0250823 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0631804 s, 830 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370761 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0679846 s, 771 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0593947 s, 883 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0432734 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306127 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0672249 s, 780 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0501644 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0555431 s, 944 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0604559 s, 867 MB/s
52428800 bytes (52 MB) copied, 0.0721039 s, 727 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0885652 s, 592 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0732278 s, 716 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0369137 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0478958 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0536056 s, 978 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0524579 s, 999 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0339138 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0310314 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0232611 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0251808 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0279379 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0365789 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0253225 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0252089 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0444639 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0267959 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0231968 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0227835 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0451394 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0335062 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0383804 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.029116 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0303008 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0701607 s, 747 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0401581 s, 1.3 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101014 s, 519 MB/s
52428800 bytes (52 MB) copied, 0.0802314 s, 653 MB/s
52428800 bytes (52 MB) copied, 0.088564 s, 592 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0377309 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0482968 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0371061 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102699 s, 511 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0617134 s, 850 MB/s
52428800 bytes (52 MB) copied, 0.101294 s, 518 MB/s
52428800 bytes (52 MB) copied, 0.0538492 s, 974 MB/s
52428800 bytes (52 MB) copied, 0.0543622 s, 964 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0778915 s, 673 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0357113 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0264909 s, 2.0 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0635334 s, 825 MB/s
52428800 bytes (52 MB) copied, 0.0897759 s, 584 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.163264 s, 321 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127892 s, 410 MB/s
52428800 bytes (52 MB) copied, 0.106284 s, 493 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0857541 s, 611 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0894027 s, 586 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0647195 s, 810 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0791014 s, 663 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0679276 s, 772 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0532275 s, 985 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0662679 s, 791 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0584196 s, 897 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0499269 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0565207 s, 928 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0374377 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0556019 s, 943 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0451405 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0236631 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0295053 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0447457 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0426908 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0714192 s, 734 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0569891 s, 920 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0694501 s, 755 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0403304 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.087294 s, 601 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0297768 s, 1.8 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0906316 s, 578 MB/s
52428800 bytes (52 MB) copied, 0.0963772 s, 544 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0459585 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0379526 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0527966 s, 993 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0284904 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.025384 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0310553 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.029427 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0801573 s, 654 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115188 s, 455 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0557712 s, 940 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0576308 s, 910 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0407673 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0773031 s, 678 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522294 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488033 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0688475 s, 762 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113376 s, 462 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0884062 s, 593 MB/s
52428800 bytes (52 MB) copied, 0.0793134 s, 661 MB/s
52428800 bytes (52 MB) copied, 0.0840748 s, 624 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0837318 s, 626 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.1797 s, 292 MB/s
52428800 bytes (52 MB) copied, 0.0921929 s, 569 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.237839 s, 220 MB/s
52428800 bytes (52 MB) copied, 0.21798 s, 241 MB/s
52428800 bytes (52 MB) copied, 0.266954 s, 196 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.132969 s, 394 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103399 s, 507 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.227789 s, 230 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.174559 s, 300 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.30629 s, 171 MB/s
52428800 bytes (52 MB) copied, 0.0751758 s, 697 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109308 s, 480 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112058 s, 468 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0561044 s, 934 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.172518 s, 304 MB/s
52428800 bytes (52 MB) copied, 0.0682647 s, 768 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0507419 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0673341 s, 779 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.045134 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0832716 s, 630 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0253543 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0975336 s, 538 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0229739 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.057928 s, 905 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0647763 s, 809 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0208955 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0231134 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0296191 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0557683 s, 940 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0540706 s, 970 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0993471 s, 528 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0762522 s, 688 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0901053 s, 582 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0500998 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0541531 s, 968 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138709 s, 378 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151737 s, 346 MB/s
52428800 bytes (52 MB) copied, 0.0959759 s, 546 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10375 s, 505 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0887558 s, 591 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145534 s, 360 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127712 s, 411 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116987 s, 448 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117295 s, 447 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.139886 s, 375 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0955721 s, 549 MB/s
, 0.10081 s, 520 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.14099 s, 372 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0883178 s, 594 MB/s
52428800 bytes (52 MB) copied, 0.11098 s, 472 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.161132 s, 325 MB/s
52428800 bytes (52 MB) copied, 0.130281 s, 402 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0545076 s, 962 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.162665 s, 322 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100883 s, 520 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.155102 s, 338 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0858396 s, 611 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.118434 s, 443 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0869621 s, 603 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0195218 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0791674 s, 662 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0859943 s, 610 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153 s, 343 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0346379 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0391019 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0596877 s, 878 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0218628 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102237 s, 513 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0355266 s, 1.5 GB/s
, 0.040897 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0286334 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0496157 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0290731 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0635112 s, 826 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0860511 s, 609 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0825897 s, 635 MB/s
52428800 bytes (52 MB) copied, 0.176236 s, 297 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0592185 s, 885 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149317 s, 351 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.141186 s, 371 MB/s
52428800 bytes (52 MB) copied, 0.176609 s, 297 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116088 s, 452 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126828 s, 413 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113566 s, 462 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11453 s, 458 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.22318 s, 235 MB/s
52428800 bytes (52 MB) copied, 0.292105 s, 179 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.261133 s, 201 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11935 s, 439 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.184564 s, 284 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.146388 s, 358 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0918383 s, 571 MB/s
52428800 bytes (52 MB) copied, 0.107815 s, 486 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0882455 s, 594 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.186118 s, 282 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.1694 s, 309 MB/s
52428800 bytes (52 MB) copied, 0.127956 s, 410 MB/s
52428800 bytes (52 MB) copied, 0.158317 s, 331 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.200862 s, 261 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.191077 s, 274 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.166689 s, 315 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103364 s, 507 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.191068 s, 274 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0916107 s, 572 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0232357 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0257183 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0326639 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0223214 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0313527 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352857 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0309156 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0495998 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0395579 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0566561 s, 925 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0578782 s, 906 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.073149 s, 717 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112082 s, 468 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.133599 s, 392 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131166 s, 400 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149648 s, 350 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.165916 s, 316 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.171367 s, 306 MB/s
, 0.140869 s, 372 MB/s
52428800 bytes (52 MB) copied, 0.145022 s, 362 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131884 s, 398 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13151 s, 399 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.24885 s, 211 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.243376 s, 215 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153513 s, 342 MB/s
52428800 bytes (52 MB) copied, 0.149335 s, 351 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0841831 s, 623 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12136 s, 432 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.213391 s, 246 MB/s
52428800 bytes (52 MB) copied, 0.10063 s, 521 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.163385 s, 321 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0269503 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.200571 s, 261 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120787 s, 434 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.176248 s, 297 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.254057 s, 206 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145098 s, 361 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.163408 s, 321 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.175114 s, 299 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153305 s, 342 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0263073 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0688033 s, 762 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.123316 s, 425 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0954326 s, 549 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.095004 s, 552 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0451969 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0646714 s, 811 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0429705 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0433984 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.041575 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0858494 s, 611 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0509698 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0341914 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0314278 s, 1.7 GB/s
52428800 bytes (52 MB) copied, 0.0978873 s, 536 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0314415 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0666469 s, 787 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0942216 s, 556 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0329727 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0643432 s, 815 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0248792 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376367 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.036432 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0424224 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0635471 s, 825 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102463 s, 512 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.1179 s, 445 MB/s
, 0.119244 s, 440 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116229 s, 451 MB/s
52428800 bytes (52 MB) copied, 0.111394 s, 471 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.139368 s, 376 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10592 s, 495 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102999 s, 509 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117794 s, 445 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136126 s, 385 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110093 s, 476 MB/s
52428800 bytes (52 MB) copied, 0.0682142 s, 769 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0657119 s, 798 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0750311 s, 699 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.150002 s, 350 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.125749 s, 417 MB/s
52428800 bytes (52 MB) copied, 0.204961 s, 256 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.118789 s, 441 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.185462 s, 283 MB/s
52428800 bytes (52 MB) copied, 0.106546 s, 492 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.154607 s, 339 MB/s
52428800 bytes (52 MB) copied, 0.152978 s, 343 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522425 s, 1.0 GB/s
52428800 bytes (52 MB) copied, 0.0512704 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.146892 s, 357 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.206864 s, 253 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0560631 s, 935 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.245979 s, 213 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.200098 s, 262 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119693 s, 438 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.154019 s, 340 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.208984 s, 251 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0980077 s, 535 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0718771 s, 729 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153093 s, 342 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.132932 s, 394 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.18044 s, 291 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0683036 s, 768 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145186 s, 361 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0542295 s, 967 MB/s
52428800 bytes (52 MB) copied, 0.0563572 s, 930 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.04982 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.132696 s, 395 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.129036 s, 406 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10407 s, 504 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.056164 s, 933 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0626202 s, 837 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.026527 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0729619 s, 719 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0623482 s, 841 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0560961 s, 935 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0586229 s, 894 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0461255 s, 1.1 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.043991 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0383755 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0537627 s, 975 MB/s
52428800 bytes (52 MB) copied, 0.0552717 s, 949 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0408836 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.021244 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0283315 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.03456 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0296976 s, 1.8 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112218 s, 467 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111724 s, 469 MB/s
52428800 bytes (52 MB) copied, 0.102308 s, 512 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0891186 s, 588 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102746 s, 510 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107245 s, 489 MB/s
52428800 bytes (52 MB) copied, 0.162845 s, 322 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.163736 s, 320 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102624 s, 511 MB/s
52428800 bytes (52 MB) copied, 0.162409 s, 323 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.134906 s, 389 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117976 s, 444 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126035 s, 416 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108923 s, 481 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.143238 s, 366 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11613 s, 451 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.187116 s, 280 MB/s
52428800 bytes (52 MB) copied, 0.124217 s, 422 MB/s
52428800 bytes (52 MB) copied, 0.0908293 s, 577 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0609814 s, 860 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120575 s, 435 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311339 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.049394 s, 1.1 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0648338 s, 809 MB/s
52428800 bytes (52 MB) copied, 0.0616961 s, 850 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0502706 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0564917 s, 928 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0226649 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0209995 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0688566 s, 761 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0203864 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0456301 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0195873 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0471426 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0299404 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0288977 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258682 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0690514 s, 759 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0220273 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0506359 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.035897 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.134997 s, 388 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0933118 s, 562 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.134827 s, 389 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127279 s, 412 MB/s
52428800 bytes (52 MB) copied, 0.149585 s, 350 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.192826 s, 272 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.140363 s, 374 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.125327 s, 418 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.16813 s, 312 MB/s
52428800 bytes (52 MB) copied, 0.162162 s, 323 MB/s
52428800 bytes (52 MB) copied, 0.122447 s, 428 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.133592 s, 392 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.146628 s, 358 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113146 s, 463 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.147855 s, 355 MB/s
, 0.129806 s, 404 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.130832 s, 401 MB/s
, 0.143248 s, 366 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138116 s, 380 MB/s
52428800 bytes (52 MB) copied, 0.124667 s, 421 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101975 s, 514 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.141861 s, 370 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128551 s, 408 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110611 s, 474 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0955449 s, 549 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0514363 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370737 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.078865 s, 665 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0417176 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0384486 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102683 s, 511 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0586286 s, 894 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0404218 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0590718 s, 888 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0724553 s, 724 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0377655 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0526431 s, 996 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15714 s, 334 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.123825 s, 423 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0795104 s, 659 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121058 s, 433 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.216229 s, 242 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.186994 s, 280 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.219372 s, 239 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.269191 s, 195 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.425239 s, 123 MB/s
52428800 bytes (52 MB) copied, 0.307701 s, 170 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.351425 s, 149 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.349547 s, 150 MB/s
52428800 bytes (52 MB) copied, 0.369176 s, 142 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.351954 s, 149 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.313823 s, 167 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.311345 s, 168 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.4248 s, 123 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.324118 s, 162 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.17735 s, 296 MB/s
52428800 bytes (52 MB) copied, 0.351411 s, 149 MB/s
52428800 bytes (52 MB) copied, 0.344353 s, 152 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.161574 s, 324 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.216012 s, 243 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.54567 s, 96.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.187677 s, 279 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.409632 s, 128 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.197431 s, 266 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.354471 s, 148 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.353024 s, 149 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.33531 s, 156 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.16798 s, 312 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.180188 s, 291 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0585656 s, 895 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.186815 s, 281 MB/s
52428800 bytes (52 MB) copied, 0.4274 s, 123 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0746705 s, 702 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.560838 s, 93.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10572 s, 496 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0437099 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0261525 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0459479 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.139906 s, 375 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.045297 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0423756 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.05085 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0467238 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0310242 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0248549 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0840024 s, 624 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0720393 s, 728 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0260401 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0613578 s, 854 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0534836 s, 980 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0464121 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.147916 s, 354 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.101531 s, 516 MB/s
52428800 bytes (52 MB) copied, 0.136411 s, 384 MB/s
52428800 bytes (52 MB) copied, 0.118273 s, 443 MB/s
52428800 bytes (52 MB) copied, 0.174399 s, 301 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151785 s, 345 MB/s
52428800 bytes (52 MB) copied, 0.136636 s, 384 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.152787 s, 343 MB/s
52428800 bytes (52 MB) copied, 0.14255 s, 368 MB/s
52428800 bytes (52 MB) copied, 0.234537 s, 224 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.14587 s, 359 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.167963 s, 312 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.169126 s, 310 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.144909 s, 362 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.195503 s, 268 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145024 s, 362 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.223978 s, 234 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.25983 s, 202 MB/s
52428800 bytes (52 MB) copied, 0.168132 s, 312 MB/s
52428800 bytes (52 MB) copied, 0.0948592 s, 553 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0954477 s, 549 MB/s
, 0.135661 s, 386 MB/s
52428800 bytes (52 MB) copied, 0.119055 s, 440 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.073973 s, 709 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.275301 s, 190 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.191623 s, 274 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101325 s, 517 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10423 s, 503 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137353 s, 382 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.070652 s, 742 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468986 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0864429 s, 607 MB/s
52428800 bytes (52 MB) copied, 0.131125 s, 400 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0723559 s, 725 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.159513 s, 329 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.161299 s, 325 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.091766 s, 571 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.165453 s, 317 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.241827 s, 217 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.194378 s, 270 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.216102 s, 243 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111765 s, 469 MB/s
52428800 bytes (52 MB) copied, 0.278833 s, 188 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124777 s, 420 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.248674 s, 211 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0918331 s, 571 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115646 s, 453 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103987 s, 504 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128196 s, 409 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10003 s, 524 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108633 s, 483 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.341122 s, 154 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.236293 s, 222 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.254343 s, 206 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.118236 s, 443 MB/s
52428800 bytes (52 MB) copied, 0.241444 s, 217 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117943 s, 445 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.242956 s, 216 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.358773 s, 146 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.193179 s, 271 MB/s
52428800 bytes (52 MB) copied, 0.202108 s, 259 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.209969 s, 250 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.249042 s, 211 MB/s
52428800 bytes (52 MB) copied, 0.190285 s, 276 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0925242 s, 567 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0634616 s, 826 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106092 s, 494 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0763605 s, 687 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0621864 s, 843 MB/s
, 0.2007 s, 261 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119588 s, 438 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15545 s, 337 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.17491 s, 300 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.169783 s, 309 MB/s
52428800 bytes (52 MB) copied, 0.250743 s, 209 MB/s
52428800 bytes (52 MB) copied, 0.270598 s, 194 MB/s
52428800 bytes (52 MB) copied, 0.242797 s, 216 MB/s
52428800 bytes (52 MB) copied, 0.207001 s, 253 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.188786 s, 278 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.182928 s, 287 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.228203 s, 230 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.306746 s, 171 MB/s
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.275595 s, 190 MB/s
50+0 records in
50+0 records out
, 0.344222 s, 152 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.308705 s, 170 MB/s
52428800 bytes (52 MB) copied, 0.254247 s, 206 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.326307 s, 161 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.338942 s, 155 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.22131 s, 237 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.236536 s, 222 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.338582 s, 155 MB/s
52428800 bytes (52 MB) copied, 0.264951 s, 198 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.219513 s, 239 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.258977 s, 202 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.20464 s, 256 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.266811 s, 197 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115917 s, 452 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142843 s, 367 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.234819 s, 223 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.253204 s, 207 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0578386 s, 906 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.328308 s, 160 MB/s
52428800 bytes (52 MB) copied, 0.417444 s, 126 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.132663 s, 395 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.175175 s, 299 MB/s
52428800 bytes (52 MB) copied, 0.0868513 s, 604 MB/s
, 0.231879 s, 226 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.32686 s, 160 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.216103 s, 243 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.181773 s, 288 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.217957 s, 241 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.214191 s, 245 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.090979 s, 576 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.223268 s, 235 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.303916 s, 173 MB/s
52428800 bytes (52 MB) copied, 0.179306 s, 292 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.345816 s, 152 MB/s
52428800 bytes (52 MB) copied, 0.254366 s, 206 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.225757 s, 232 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.235973 s, 222 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.286375 s, 183 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.250955 s, 209 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.328161 s, 160 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.294551 s, 178 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.266047 s, 197 MB/s
52428800 bytes (52 MB) copied, 0.364091 s, 144 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.346826 s, 151 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.264122 s, 199 MB/s
, 0.14099 s, 372 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.167793 s, 312 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.114274 s, 459 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.294148 s, 178 MB/s
52428800 bytes (52 MB) copied, 0.33888 s, 155 MB/s
52428800 bytes (52 MB) copied, 0.313921 s, 167 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.367725 s, 143 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.26391 s, 199 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.257138 s, 204 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.23235 s, 226 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0574012 s, 913 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.043076 s, 1.2 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0256966 s, 2.0 GB/s
52428800 bytes (52 MB) copied, 0.151767 s, 345 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0304767 s, 1.7 GB/s
52428800 bytes (52 MB) copied, 0.0787923 s, 665 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142984 s, 367 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0530891 s, 988 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113769 s, 461 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13676 s, 383 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0244624 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.292127 s, 179 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0857669 s, 611 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0786131 s, 667 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0973864 s, 538 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0452697 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0699082 s, 750 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0988615 s, 530 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126044 s, 416 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122164 s, 429 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0795913 s, 659 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.201857 s, 260 MB/s
52428800 bytes (52 MB) copied, 0.159231 s, 329 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.195949 s, 268 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.166194 s, 315 MB/s
52428800 bytes (52 MB) copied, 0.239326 s, 219 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.291428 s, 180 MB/s
52428800 bytes (52 MB) copied, 0.2324 s, 226 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.197266 s, 266 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.228934 s, 229 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.191505 s, 274 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.226061 s, 232 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.225469 s, 233 MB/s
52428800 bytes (52 MB) copied, 0.256195 s, 205 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.377675 s, 139 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.297548 s, 176 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.217697 s, 241 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.243426 s, 215 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.229004 s, 229 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.376492 s, 139 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.210635 s, 249 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.347338 s, 151 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.466477 s, 112 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.349519 s, 150 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.446996 s, 117 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.17063 s, 307 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.254714 s, 206 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.225487 s, 233 MB/s
52428800 bytes (52 MB) copied, 0.389859 s, 134 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.167379 s, 313 MB/s
52428800 bytes (52 MB) copied, 0.394074 s, 133 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.247053 s, 212 MB/s
52428800 bytes (52 MB) copied, 0.0498315 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.271462 s, 193 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.194845 s, 269 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0578486 s, 906 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.174802 s, 300 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.210141 s, 249 MB/s
, 0.099262 s, 528 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121752 s, 431 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0918066 s, 571 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0519807 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0344826 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0275359 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0708089 s, 740 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0809841 s, 647 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.08881 s, 590 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0638735 s, 821 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0665038 s, 788 MB/s
52428800 bytes (52 MB) copied, 0.132637 s, 395 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124291 s, 422 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.211703 s, 248 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.172248 s, 304 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0906975 s, 578 MB/s
52428800 bytes (52 MB) copied, 0.194025 s, 270 MB/s
52428800 bytes (52 MB) copied, 0.119098 s, 440 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.167324 s, 313 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153252 s, 342 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.22465 s, 233 MB/s
, 0.166221 s, 315 MB/s
52428800 bytes (52 MB) copied, 0.209542 s, 250 MB/s
52428800 bytes (52 MB) copied, 0.240164 s, 218 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.288966 s, 181 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.275177 s, 191 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.26626 s, 197 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.193317 s, 271 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117423 s, 446 MB/s
52428800 bytes (52 MB) copied, 0.0887956 s, 590 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.224006 s, 234 MB/s
, 0.295895 s, 177 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.223291 s, 235 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.327619 s, 160 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108049 s, 485 MB/s
52428800 bytes (52 MB) copied, 0.282372 s, 186 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149835 s, 350 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0278643 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11378 s, 461 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104555 s, 501 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.28716 s, 183 MB/s
52428800 bytes (52 MB) copied, 0.107598 s, 487 MB/s
50+0 records in
50+0 records out
, 0.161984 s, 324 MB/s
52428800 bytes (52 MB) copied, 0.259993 s, 202 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0821908 s, 638 MB/s
52428800 bytes (52 MB) copied, 0.0463308 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.121146 s, 433 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0486912 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0362589 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.030658 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0224016 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0406849 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336543 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0328531 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.027973 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336685 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0251848 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382841 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.053814 s, 974 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0543812 s, 964 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370004 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0640922 s, 818 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0666317 s, 787 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0580314 s, 903 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0409281 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0826392 s, 634 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0196601 s, 2.7 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.098593 s, 532 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0233947 s, 2.2 GB/s
52428800 bytes (52 MB) copied, 0.105778 s, 496 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.182543 s, 287 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.179737 s, 292 MB/s
52428800 bytes (52 MB) copied, 0.211426 s, 248 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.195691 s, 268 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.21411 s, 245 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.260107 s, 202 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.235287 s, 223 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.332341 s, 158 MB/s
52428800 bytes (52 MB) copied, 0.384403 s, 136 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.29349 s, 179 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.235374 s, 223 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.346985 s, 151 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.247638 s, 212 MB/s
52428800 bytes (52 MB) copied, 0.383105 s, 137 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.354191 s, 148 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.320368 s, 164 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.407338 s, 129 MB/s
52428800 bytes (52 MB) copied, 0.167533 s, 313 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.440957 s, 119 MB/s
52428800 bytes (52 MB) copied, 0.17012 s, 308 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.261095 s, 201 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.207477 s, 253 MB/s
52428800 bytes (52 MB) copied, 0.262432 s, 200 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.267647 s, 196 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.230902 s, 227 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.526864 s, 99.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376731 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.422514 s, 124 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.445654 s, 118 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0846361 s, 619 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0858268 s, 611 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113524 s, 462 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.293303 s, 179 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.330788 s, 158 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0191137 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.251506 s, 208 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.132512 s, 396 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.588923 s, 89.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.312419 s, 168 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.595905 s, 88.0 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.118072 s, 444 MB/s
, 0.237481 s, 221 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107991 s, 485 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.163841 s, 320 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0776019 s, 676 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0941773 s, 557 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.246483 s, 213 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.166417 s, 315 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.184336 s, 284 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.37801 s, 139 MB/s
52428800 bytes (52 MB) copied, 0.166173 s, 316 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.202956 s, 258 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.207789 s, 252 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.364473 s, 144 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.253761 s, 207 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.165178 s, 317 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.209812 s, 250 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.288352 s, 182 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.254636 s, 206 MB/s
, 0.255893 s, 205 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115569 s, 454 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115574 s, 454 MB/s
52428800 bytes (52 MB) copied, 0.305836 s, 171 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.22508 s, 233 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.221455 s, 237 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.213503 s, 246 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.260627 s, 201 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.242382 s, 216 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.245429 s, 214 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.327514 s, 160 MB/s
52428800 bytes (52 MB) copied, 0.167695 s, 313 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.306321 s, 171 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.186571 s, 281 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.182591 s, 287 MB/s
52428800 bytes (52 MB) copied, 0.174883 s, 300 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109531 s, 479 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.401418 s, 131 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.152664 s, 343 MB/s
52428800 bytes (52 MB) copied, 0.129978 s, 403 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.377945 s, 139 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.362093 s, 145 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.1231 s, 426 MB/s
52428800 bytes (52 MB) copied, 0.187624 s, 279 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142146 s, 369 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.120047 s, 437 MB/s
, 0.144513 s, 363 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.107975 s, 486 MB/s
, 0.117741 s, 445 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0472265 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0943969 s, 555 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0724551 s, 724 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124665 s, 421 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.214588 s, 244 MB/s
, 0.230535 s, 227 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.196159 s, 267 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.185759 s, 282 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.301779 s, 174 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.182336 s, 288 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.219736 s, 239 MB/s
52428800 bytes (52 MB) copied, 0.231215 s, 227 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.237124 s, 221 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.305708 s, 171 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.201326 s, 260 MB/s
52428800 bytes (52 MB) copied, 0.315508 s, 166 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.290022 s, 181 MB/s
52428800 bytes (52 MB) copied, 0.297103 s, 176 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.247314 s, 212 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.296266 s, 177 MB/s
52428800 bytes (52 MB) copied, 0.264938 s, 198 MB/s
52428800 bytes (52 MB) copied, 0.277232 s, 189 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.211064 s, 248 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.226242 s, 232 MB/s
52428800 bytes (52 MB) copied, 0.321205 s, 163 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.472357 s, 111 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.320697 s, 163 MB/s
52428800 bytes (52 MB) copied, 0.220733 s, 238 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.333154 s, 157 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.286653 s, 183 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.238342 s, 220 MB/s
50+0 records in
50+0 records out
, 0.447251 s, 117 MB/s
52428800 bytes (52 MB) copied, 0.443825 s, 118 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.580499 s, 90.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.269568 s, 194 MB/s
52428800 bytes (52 MB) copied, 0.314889 s, 166 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.192417 s, 272 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0986793 s, 531 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0813808 s, 644 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.549325 s, 95.4 MB/s
52428800 bytes (52 MB) copied, 0.0420552 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0601138 s, 872 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136641 s, 384 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0856111 s, 612 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0386477 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0397746 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352804 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0226416 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0953478 s, 550 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0975654 s, 537 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0253777 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0495214 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0326481 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0623724 s, 841 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0340893 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0598826 s, 876 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0473021 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0682041 s, 769 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0547793 s, 957 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0333539 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382746 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0511251 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0268462 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0716032 s, 732 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0384143 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.064854 s, 808 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0986841 s, 531 MB/s
52428800 bytes (52 MB) copied, 0.123627 s, 424 MB/s
52428800 bytes (52 MB) copied, 0.190909 s, 275 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.222073 s, 236 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.283431 s, 185 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.277256 s, 189 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.24544 s, 214 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.350141 s, 150 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.200198 s, 262 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.282966 s, 185 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.314 s, 167 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.369367 s, 142 MB/s
52428800 bytes (52 MB) copied, 0.172051 s, 305 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.364474 s, 144 MB/s
52428800 bytes (52 MB) copied, 0.192093 s, 273 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.408374 s, 128 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.341115 s, 154 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.3553 s, 148 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.360285 s, 146 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.341714 s, 153 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.400762 s, 131 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.368958 s, 142 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.273681 s, 192 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.309778 s, 169 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.311871 s, 168 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.312011 s, 168 MB/s
52428800 bytes (52 MB) copied, 0.594217 s, 88.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.249285 s, 210 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.282753 s, 185 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.252446 s, 208 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138364 s, 379 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.1364 s, 384 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15511 s, 338 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0929877 s, 564 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0895128 s, 586 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.303842 s, 173 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0514038 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149929 s, 350 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.292024 s, 180 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0976208 s, 537 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.043894 s, 1.2 GB/s
52428800 bytes (52 MB) copied, 0.0529338 s, 990 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112696 s, 465 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.236121 s, 222 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.161053 s, 326 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122251 s, 429 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0962674 s, 545 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115373 s, 454 MB/s
52428800 bytes (52 MB) copied, 0.261881 s, 200 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110013 s, 477 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11306 s, 464 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.170417 s, 308 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100683 s, 521 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.114766 s, 457 MB/s
52428800 bytes (52 MB) copied, 0.226333 s, 232 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.30266 s, 173 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.169252 s, 310 MB/s
52428800 bytes (52 MB) copied, 0.308405 s, 170 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.36572 s, 143 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.234688 s, 223 MB/s
52428800 bytes (52 MB) copied, 0.315626 s, 166 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.19636 s, 267 MB/s
52428800 bytes (52 MB) copied, 0.103066 s, 509 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.318498 s, 165 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104596 s, 501 MB/s
52428800 bytes (52 MB) copied, 0.136657 s, 384 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.252618 s, 208 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.257427 s, 204 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.300225 s, 175 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.229261 s, 229 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.162608 s, 322 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.185915 s, 282 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.156038 s, 336 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.204852 s, 256 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.182632 s, 287 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.16499 s, 318 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.108548 s, 483 MB/s
52428800 bytes (52 MB) copied, 0.373513 s, 140 MB/s
52428800 bytes (52 MB) copied, 0.332048 s, 158 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.287313 s, 182 MB/s
52428800 bytes (52 MB) copied, 0.421015 s, 125 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488084 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0540526 s, 970 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.226958 s, 231 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.187327 s, 280 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.181794 s, 288 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.468722 s, 112 MB/s
52428800 bytes (52 MB) copied, 0.239366 s, 219 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.219131 s, 239 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.295836 s, 177 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.143275 s, 366 MB/s
52428800 bytes (52 MB) copied, 0.160386 s, 327 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.253858 s, 207 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.240353 s, 218 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.190931 s, 275 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.204899 s, 256 MB/s
, 0.111285 s, 471 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.197954 s, 265 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.346531 s, 151 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.150788 s, 348 MB/s
52428800 bytes (52 MB) copied, 0.256253 s, 205 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.31714 s, 165 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.189682 s, 276 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.188055 s, 279 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.192579 s, 272 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.332759 s, 158 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.317452 s, 165 MB/s
52428800 bytes (52 MB) copied, 0.321311 s, 163 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.261368 s, 201 MB/s
52428800 bytes (52 MB) copied, 0.294359 s, 178 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.198225 s, 264 MB/s
52428800 bytes (52 MB) copied, 0.269181 s, 195 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.433328 s, 121 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.260264 s, 201 MB/s
52428800 bytes (52 MB) copied, 0.216779 s, 242 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.49218 s, 107 MB/s
52428800 bytes (52 MB) copied, 0.41331 s, 127 MB/s
52428800 bytes (52 MB) copied, 0.383077 s, 137 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.279355 s, 188 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.270429 s, 194 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.290224 s, 181 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.404874 s, 129 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.445618 s, 118 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.292551 s, 179 MB/s
52428800 bytes (52 MB) copied, 0.409682 s, 128 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.312795 s, 168 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.306306 s, 171 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.401529 s, 131 MB/s
52428800 bytes (52 MB) copied, 0.35665 s, 147 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.419701 s, 125 MB/s
52428800 bytes (52 MB) copied, 0.274412 s, 191 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.290994 s, 180 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.590163 s, 88.8 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.42454 s, 123 MB/s
52428800 bytes (52 MB) copied, 0.385184 s, 136 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.336531 s, 156 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.536458 s, 97.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.624013 s, 84.0 MB/s
52428800 bytes (52 MB) copied, 0.295757 s, 177 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.293287 s, 179 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.238915 s, 219 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.280235 s, 187 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.191731 s, 273 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.398685 s, 132 MB/s
52428800 bytes (52 MB) copied, 0.218549 s, 240 MB/s
52428800 bytes (52 MB) copied, 0.317751 s, 165 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.211607 s, 248 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0936965 s, 560 MB/s
, 0.345452 s, 152 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113687 s, 461 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.37114 s, 141 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.143782 s, 365 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.402867 s, 130 MB/s
52428800 bytes (52 MB) copied, 0.102625 s, 511 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100184 s, 523 MB/s
52428800 bytes (52 MB) copied, 0.411341 s, 127 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.132232 s, 396 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0748 s, 701 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145659 s, 360 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.180217 s, 291 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.341472 s, 154 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.286903 s, 183 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.313572 s, 167 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.330519 s, 159 MB/s
52428800 bytes (52 MB) copied, 0.370837 s, 141 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.29642 s, 177 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.629635 s, 83.3 MB/s
52428800 bytes (52 MB) copied, 0.461175 s, 114 MB/s
, 0.397264 s, 132 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.326543 s, 161 MB/s
52428800 bytes (52 MB) copied, 0.425941 s, 123 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.305124 s, 172 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.467497 s, 112 MB/s
52428800 bytes (52 MB) copied, 0.302387 s, 173 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.401301 s, 131 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.400618 s, 131 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.532007 s, 98.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.462828 s, 113 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.4178 s, 125 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
, 0.262077 s, 200 MB/s
52428800 bytes (52 MB) copied, 0.4449 s, 118 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.590412 s, 88.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.175738 s, 298 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.135964 s, 386 MB/s
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.593599 s, 88.3 MB/s
, 0.267829 s, 196 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.597967 s, 87.7 MB/s
52428800 bytes (52 MB) copied, 0.522337 s, 100 MB/s
52428800 bytes (52 MB) copied, 0.667 s, 78.6 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.552833 s, 94.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.174387 s, 301 MB/s
52428800 bytes (52 MB) copied, 0.504309 s, 104 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0669149 s, 784 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0839074 s, 625 MB/s
52428800 bytes (52 MB) copied, 0.31158 s, 168 MB/s
, 0.234108 s, 224 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0808606 s, 648 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0424585 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.208561 s, 251 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.206862 s, 253 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.202574 s, 259 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.231743 s, 226 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.242054 s, 217 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.20081 s, 261 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112254 s, 467 MB/s
52428800 bytes (52 MB) copied, 0.29483 s, 178 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0811012 s, 646 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.227391 s, 231 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.253692 s, 207 MB/s
52428800 bytes (52 MB) copied, 0.147988 s, 354 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.199085 s, 263 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.182226 s, 288 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.264214 s, 198 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.181933 s, 288 MB/s
52428800 bytes (52 MB) copied, 0.197225 s, 266 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.167882 s, 312 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.252578 s, 208 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.188196 s, 279 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.225894 s, 232 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.247837 s, 212 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.196299 s, 267 MB/s
52428800 bytes (52 MB) copied, 0.19136 s, 274 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.161897 s, 324 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.165489 s, 317 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.21924 s, 239 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.165262 s, 317 MB/s
52428800 bytes (52 MB) copied, 0.151985 s, 345 MB/s
52428800 bytes (52 MB) copied, 0.335593 s, 156 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.344718 s, 152 MB/s
52428800 bytes (52 MB) copied, 0.244521 s, 214 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12116 s, 433 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.342365 s, 153 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127517 s, 411 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0595794 s, 880 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.050245 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.172252 s, 304 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.429864 s, 122 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.340372 s, 154 MB/s
52428800 bytes (52 MB) copied, 0.227501 s, 230 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.176965 s, 296 MB/s
52428800 bytes (52 MB) copied, 0.407675 s, 129 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.197365 s, 266 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.17085 s, 307 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.325517 s, 161 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.211153 s, 248 MB/s
52428800 bytes (52 MB) copied, 0.161706 s, 324 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108936 s, 481 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.16414 s, 319 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.241858 s, 217 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.164243 s, 319 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.147273 s, 356 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.17866 s, 293 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103151 s, 508 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.275533 s, 190 MB/s
, 0.155982 s, 336 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0540068 s, 971 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0426474 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0875941 s, 599 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0936638 s, 560 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0741315 s, 707 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.371924 s, 141 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.256669 s, 204 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122633 s, 428 MB/s
52428800 bytes (52 MB) copied, 0.29412 s, 178 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126728 s, 414 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.253389 s, 207 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0537829 s, 975 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115827 s, 453 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.141972 s, 369 MB/s
, 0.109549 s, 479 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.294378 s, 178 MB/s
52428800 bytes (52 MB) copied, 0.0947638 s, 553 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0591308 s, 887 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.123737 s, 424 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0532718 s, 984 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.114507 s, 458 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0565112 s, 928 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0402336 s, 1.3 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0331021 s, 1.6 GB/s
52428800 bytes (52 MB) copied, 0.0554039 s, 946 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0681802 s, 769 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0428888 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0817752 s, 641 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0546805 s, 959 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0905575 s, 579 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12998 s, 403 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0694943 s, 754 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0606689 s, 864 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0983895 s, 533 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0250726 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.175602 s, 299 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.234054 s, 224 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.201041 s, 261 MB/s
52428800 bytes (52 MB) copied, 0.044113 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.181589 s, 289 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0665015 s, 788 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110757 s, 473 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.315098 s, 166 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.247123 s, 212 MB/s
52428800 bytes (52 MB) copied, 0.382642 s, 137 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.404015 s, 130 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.201612 s, 260 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.218643 s, 240 MB/s
52428800 bytes (52 MB) copied, 0.431235 s, 122 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.220334 s, 238 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.249725 s, 210 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.440561 s, 119 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.384042 s, 137 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.218296 s, 240 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.316154 s, 166 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.276011 s, 190 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.324305 s, 162 MB/s
52428800 bytes (52 MB) copied, 0.346973 s, 151 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.391022 s, 134 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.327895 s, 160 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.259844 s, 202 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.286189 s, 183 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122687 s, 427 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.470307 s, 111 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0941483 s, 557 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.324901 s, 161 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.501737 s, 104 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.566788 s, 92.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.346173 s, 151 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.338396 s, 155 MB/s
, 0.28771 s, 182 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.454057 s, 115 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.234595 s, 223 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.221091 s, 237 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.469595 s, 112 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.227292 s, 231 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.210225 s, 249 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149988 s, 350 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.297283 s, 176 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.163637 s, 320 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.189183 s, 277 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.221228 s, 237 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.183246 s, 286 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.174152 s, 301 MB/s
52428800 bytes (52 MB) copied, 0.16579 s, 316 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.339102 s, 155 MB/s
52428800 bytes (52 MB) copied, 0.284428 s, 184 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.408548 s, 128 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.332798 s, 158 MB/s
52428800 bytes (52 MB) copied, 0.315164 s, 166 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.28867 s, 182 MB/s
, 0.330671 s, 159 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.253043 s, 207 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.230189 s, 228 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.238911 s, 219 MB/s
52428800 bytes (52 MB) copied, 0.213563 s, 245 MB/s
52428800 bytes (52 MB) copied, 0.307139 s, 171 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.324732 s, 161 MB/s
52428800 bytes (52 MB) copied, 0.128912 s, 407 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.322092 s, 163 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.095539 s, 549 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131706 s, 398 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.20038 s, 262 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.21223 s, 247 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.2267 s, 231 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.1735 s, 302 MB/s
52428800 bytes (52 MB) copied, 0.203664 s, 257 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.154934 s, 338 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.123929 s, 423 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.192005 s, 273 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15268 s, 343 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.228685 s, 229 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137676 s, 381 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.251695 s, 208 MB/s
52428800 bytes (52 MB) copied, 0.207779 s, 252 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.167219 s, 314 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.0875477 s, 599 MB/s
52428800 bytes (52 MB) copied, 0.0723361 s, 725 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.222914 s, 235 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.293692 s, 179 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.255352 s, 205 MB/s
52428800 bytes (52 MB) copied, 0.16689 s, 314 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.291007 s, 180 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.174845 s, 300 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.197413 s, 266 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.224588 s, 233 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.240195 s, 218 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.217979 s, 241 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.219355 s, 239 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157426 s, 333 MB/s
52428800 bytes (52 MB) copied, 0.23529 s, 223 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.183222 s, 286 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.173569 s, 302 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.194863 s, 269 MB/s
, 0.188259 s, 278 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.169535 s, 309 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137215 s, 382 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0968449 s, 541 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0906225 s, 579 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.250954 s, 209 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.323633 s, 162 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0444452 s, 1.2 GB/s
52428800 bytes (52 MB) copied, 0.233969 s, 224 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.26446 s, 198 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.265473 s, 197 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109298 s, 480 MB/s
52428800 bytes (52 MB) copied, 0.129022 s, 406 MB/s
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.182496 s, 287 MB/s
52428800 bytes (52 MB) copied, 0.235725 s, 222 MB/s
, 0.247855 s, 212 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145865 s, 359 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.23054 s, 227 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0785338 s, 668 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0983365 s, 533 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126981 s, 413 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.229144 s, 229 MB/s
52428800 bytes (52 MB) copied, 0.319648 s, 164 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.272956 s, 192 MB/s
52428800 bytes (52 MB) copied, 0.254273 s, 206 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.217781 s, 241 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131554 s, 399 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.270998 s, 193 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0948608 s, 553 MB/s
52428800 bytes (52 MB) copied, 0.183689 s, 285 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.318374 s, 165 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.257604 s, 204 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.213554 s, 246 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.213243 s, 246 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.173847 s, 302 MB/s
, 0.245796 s, 213 MB/s
52428800 bytes (52 MB) copied, 0.170827 s, 307 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131768 s, 398 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.198152 s, 265 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.185084 s, 283 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.203416 s, 258 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.30765 s, 170 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.166176 s, 316 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.194571 s, 269 MB/s
, 0.158313 s, 331 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0494316 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.0738546 s, 710 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124309 s, 422 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.180698 s, 290 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157418 s, 333 MB/s
52428800 bytes (52 MB) copied, 0.18706 s, 280 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375962 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0881133 s, 595 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0792091 s, 662 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110521 s, 474 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124833 s, 420 MB/s
52428800 bytes (52 MB) copied, 0.155927 s, 336 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124241 s, 422 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.175616 s, 299 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.26823 s, 195 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.22695 s, 231 MB/s
52428800 bytes (52 MB) copied, 0.133709 s, 392 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.152812 s, 343 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.16873 s, 311 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149021 s, 352 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.229221 s, 229 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.14326 s, 366 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.261204 s, 201 MB/s
52428800 bytes (52 MB) copied, 0.175753 s, 298 MB/s
52428800 bytes (52 MB) copied, 0.163111 s, 321 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126174 s, 416 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124437 s, 421 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.27534 s, 190 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15025 s, 349 MB/s
52428800 bytes (52 MB) copied, 0.114302 s, 459 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0494726 s, 1.1 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106718 s, 491 MB/s
52428800 bytes (52 MB) copied, 0.232456 s, 226 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.148882 s, 352 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0553008 s, 948 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0734959 s, 713 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0555352 s, 944 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0635772 s, 825 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.099072 s, 529 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0479902 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0314179 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.032733 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0785498 s, 667 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0268151 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0225254 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.023911 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0739022 s, 709 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0750381 s, 699 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0643352 s, 815 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0500004 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0581358 s, 902 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0594221 s, 882 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.135776 s, 386 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145815 s, 360 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.152266 s, 344 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.183173 s, 286 MB/s
, 0.166129 s, 316 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.184589 s, 284 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15759 s, 333 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.172074 s, 305 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.168321 s, 311 MB/s
52428800 bytes (52 MB) copied, 0.228035 s, 230 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.206624 s, 254 MB/s
52428800 bytes (52 MB) copied, 0.186439 s, 281 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.216541 s, 242 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.306939 s, 171 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.232858 s, 225 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.24424 s, 215 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.203378 s, 258 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.241228 s, 217 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.212258 s, 247 MB/s
52428800 bytes (52 MB) copied, 0.144171 s, 364 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.161329 s, 325 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.29142 s, 180 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.281048 s, 187 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.189827 s, 276 MB/s
52428800 bytes (52 MB) copied, 0.280823 s, 187 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.173777 s, 302 MB/s
52428800 bytes (52 MB) copied, 0.246376 s, 213 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.259973 s, 202 MB/s
52428800 bytes (52 MB) copied, 0.262508 s, 200 MB/s
52428800 bytes (52 MB) copied, 0.156727 s, 335 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.155313 s, 338 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103936 s, 504 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142993 s, 367 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112985 s, 464 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.169193 s, 310 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.269064 s, 195 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.196726 s, 267 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.181822 s, 288 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.14712 s, 356 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.247083 s, 212 MB/s
, 0.172811 s, 303 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.197725 s, 265 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.26999 s, 194 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.360222 s, 146 MB/s
, 0.242338 s, 216 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.198375 s, 264 MB/s
52428800 bytes (52 MB) copied, 0.299732 s, 175 MB/s
52428800 bytes (52 MB) copied, 0.23953 s, 219 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.44124 s, 119 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.172747 s, 304 MB/s
52428800 bytes (52 MB) copied, 0.447551 s, 117 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.310658 s, 169 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.181234 s, 289 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.331555 s, 158 MB/s
52428800 bytes (52 MB) copied, 0.367599 s, 143 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.164707 s, 318 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.278651 s, 188 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.201919 s, 260 MB/s
52428800 bytes (52 MB) copied, 0.315402 s, 166 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.204384 s, 257 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.26279 s, 200 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.154454 s, 339 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.194494 s, 270 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.186003 s, 282 MB/s
, 0.376133 s, 139 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.134862 s, 389 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112165 s, 467 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.291073 s, 180 MB/s
52428800 bytes (52 MB) copied, 0.304873 s, 172 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.370827 s, 141 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0959321 s, 547 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.369279 s, 142 MB/s
52428800 bytes (52 MB) copied, 0.0991133 s, 529 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110606 s, 474 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.147624 s, 355 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.243373 s, 215 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.214172 s, 245 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0875922 s, 599 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.203431 s, 258 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.233694 s, 224 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.179549 s, 292 MB/s
52428800 bytes (52 MB) copied, 0.117325 s, 447 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.245886 s, 213 MB/s
52428800 bytes (52 MB) copied, 0.116041 s, 452 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0607866 s, 863 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.269206 s, 195 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0625482 s, 838 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.196905 s, 266 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.215264 s, 244 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.129842 s, 404 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109717 s, 478 MB/s
52428800 bytes (52 MB) copied, 0.128588 s, 408 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0532294 s, 985 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.069492 s, 754 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.091419 s, 573 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0816764 s, 642 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149273 s, 351 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119743 s, 438 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106395 s, 493 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0944544 s, 555 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131731 s, 398 MB/s
52428800 bytes (52 MB) copied, 0.169375 s, 310 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0678624 s, 773 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0771365 s, 680 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13527 s, 388 MB/s
52428800 bytes (52 MB) copied, 0.164763 s, 318 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0752892 s, 696 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0944942 s, 555 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.188742 s, 278 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.09373 s, 559 MB/s
52428800 bytes (52 MB) copied, 0.162741 s, 322 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.232566 s, 225 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.213799 s, 245 MB/s
52428800 bytes (52 MB) copied, 0.0979198 s, 535 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131134 s, 400 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.21332 s, 246 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145283 s, 361 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.298435 s, 176 MB/s
52428800 bytes (52 MB) copied, 0.215002 s, 244 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122795 s, 427 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.269691 s, 194 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120467 s, 435 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13593 s, 386 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.168299 s, 312 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0716867 s, 731 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.18123 s, 289 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.143147 s, 366 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11586 s, 453 MB/s
50+0 records in
50+0 records out
, 0.0613296 s, 855 MB/s
52428800 bytes (52 MB) copied, 0.121489 s, 432 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106219 s, 494 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103571 s, 506 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0482539 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11586 s, 453 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0323354 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.206719 s, 254 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0247842 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0331807 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137787 s, 381 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0588539 s, 891 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0296907 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0548982 s, 955 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0681189 s, 770 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.043778 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0408369 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0607739 s, 863 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0573445 s, 914 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0604071 s, 868 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0406688 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0440749 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0661267 s, 793 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102259 s, 513 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138912 s, 377 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0937066 s, 559 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119892 s, 437 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136964 s, 383 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145286 s, 361 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.094936 s, 552 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153099 s, 342 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.138497 s, 379 MB/s
, 0.206279 s, 254 MB/s
52428800 bytes (52 MB) copied, 0.16433 s, 319 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145148 s, 361 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0937078 s, 559 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.160507 s, 327 MB/s
52428800 bytes (52 MB) copied, 0.174104 s, 301 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0841184 s, 623 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100089 s, 524 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151177 s, 347 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.162962 s, 322 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.161494 s, 325 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109816 s, 477 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113084 s, 464 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.190506 s, 275 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100684 s, 521 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375775 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0294042 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0648375 s, 809 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0827291 s, 634 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0600772 s, 873 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0577058 s, 909 MB/s
52428800 bytes (52 MB) copied, 0.0724759 s, 723 MB/s
52428800 bytes (52 MB) copied, 0.0582283 s, 900 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0745079 s, 704 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0804155 s, 652 MB/s
52428800 bytes (52 MB) copied, 0.11919 s, 440 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0907497 s, 578 MB/s
52428800 bytes (52 MB) copied, 0.120058 s, 437 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105363 s, 498 MB/s
52428800 bytes (52 MB) copied, 0.10718 s, 489 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105556 s, 497 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0885223 s, 592 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106535 s, 492 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157084 s, 334 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0833313 s, 629 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12516 s, 419 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.129425 s, 405 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0805133 s, 651 MB/s
52428800 bytes (52 MB) copied, 0.0730561 s, 718 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0886705 s, 591 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.132551 s, 396 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0456534 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0675335 s, 776 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0293616 s, 1.8 GB/s
52428800 bytes (52 MB) copied, 0.138248 s, 379 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0439549 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0576557 s, 909 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0470715 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0712697 s, 736 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.056183 s, 933 MB/s
52428800 bytes (52 MB) copied, 0.0628915 s, 834 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.047989 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0507534 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0580532 s, 903 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0617374 s, 849 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0527161 s, 995 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0579761 s, 904 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112408 s, 466 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.057876 s, 906 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111464 s, 470 MB/s
52428800 bytes (52 MB) copied, 0.060177 s, 871 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0620126 s, 845 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0399853 s, 1.3 GB/s
52428800 bytes (52 MB) copied, 0.0878224 s, 597 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0330393 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103188 s, 508 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0869958 s, 603 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0765124 s, 685 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0964332 s, 544 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0668133 s, 785 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0758331 s, 691 MB/s
52428800 bytes (52 MB) copied, 0.0482938 s, 1.1 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0675918 s, 776 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0743222 s, 705 MB/s
52428800 bytes (52 MB) copied, 0.126499 s, 414 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0434647 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0486327 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0639234 s, 820 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0423467 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0559886 s, 936 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0273609 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308838 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.035458 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382659 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0349633 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0745723 s, 703 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0422371 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0811296 s, 646 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.0460763 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.0613677 s, 854 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0868054 s, 604 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0425242 s, 1.2 GB/s
52428800 bytes (52 MB) copied, 0.119429 s, 439 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.16372 s, 320 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136909 s, 383 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116493 s, 450 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.143826 s, 365 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0992617 s, 528 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.14669 s, 357 MB/s
52428800 bytes (52 MB) copied, 0.141987 s, 369 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.147694 s, 355 MB/s
52428800 bytes (52 MB) copied, 0.165484 s, 317 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0769731 s, 681 MB/s
52428800 bytes (52 MB) copied, 0.0952346 s, 551 MB/s
52428800 bytes (52 MB) copied, 0.236273 s, 222 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108921 s, 481 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.125388 s, 418 MB/s
52428800 bytes (52 MB) copied, 0.140847 s, 372 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15277 s, 343 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.162772 s, 322 MB/s
52428800 bytes (52 MB) copied, 0.0917681 s, 571 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0635847 s, 825 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0584185 s, 897 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.130314 s, 402 MB/s
52428800 bytes (52 MB) copied, 0.0463452 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.135044 s, 388 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.155743 s, 337 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.1029 s, 510 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0883043 s, 594 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0778712 s, 673 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0625357 s, 838 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0253392 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0557037 s, 941 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0463516 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0787605 s, 666 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0596164 s, 879 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0789264 s, 664 MB/s
, 0.101071 s, 519 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0752324 s, 697 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112879 s, 464 MB/s
52428800 bytes (52 MB) copied, 0.0736875 s, 712 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100772 s, 520 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0681722 s, 769 MB/s
52428800 bytes (52 MB) copied, 0.0536716 s, 977 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0851496 s, 616 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100862 s, 520 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.038645 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.134955 s, 388 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.140319 s, 374 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0871536 s, 602 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0745333 s, 703 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0386372 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0292843 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.063629 s, 824 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0437151 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0220685 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0249165 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0881755 s, 595 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0674545 s, 777 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.036778 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0356216 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0244642 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0763363 s, 687 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0650056 s, 807 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0795632 s, 659 MB/s
, 0.102288 s, 513 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.152324 s, 344 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120649 s, 435 MB/s
52428800 bytes (52 MB) copied, 0.141539 s, 370 MB/s
52428800 bytes (52 MB) copied, 0.123355 s, 425 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124454 s, 421 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112424 s, 466 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157708 s, 332 MB/s
52428800 bytes (52 MB) copied, 0.127371 s, 412 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127581 s, 411 MB/s
52428800 bytes (52 MB) copied, 0.127791 s, 410 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.155341 s, 338 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.08795 s, 596 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.11425 s, 459 MB/s
, 0.0822721 s, 637 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10178 s, 515 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0882461 s, 594 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.187353 s, 280 MB/s
52428800 bytes (52 MB) copied, 0.196122 s, 267 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0912491 s, 575 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.168469 s, 311 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.204367 s, 257 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151466 s, 346 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138594 s, 378 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0770081 s, 681 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0585309 s, 896 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0430595 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.078867 s, 665 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0526334 s, 996 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0776659 s, 675 MB/s
52428800 bytes (52 MB) copied, 0.10726 s, 489 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10294 s, 509 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113809 s, 461 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100815 s, 520 MB/s
52428800 bytes (52 MB) copied, 0.109605 s, 478 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13298 s, 394 MB/s
52428800 bytes (52 MB) copied, 0.179068 s, 293 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0995786 s, 527 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.221525 s, 237 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126621 s, 414 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0719614 s, 729 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0842092 s, 623 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.147616 s, 355 MB/s
52428800 bytes (52 MB) copied, 0.0659159 s, 795 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0989646 s, 530 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107796 s, 486 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.09378 s, 559 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0511084 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.152897 s, 343 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0802825 s, 653 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0623093 s, 841 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0661285 s, 793 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.05378 s, 975 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0753633 s, 696 MB/s
52428800 bytes (52 MB) copied, 0.0665582 s, 788 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0750166 s, 699 MB/s
52428800 bytes (52 MB) copied, 0.0566386 s, 926 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0436727 s, 1.2 GB/s
52428800 bytes (52 MB) copied, 0.128602 s, 408 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0625142 s, 839 MB/s
52428800 bytes (52 MB) copied, 0.121176 s, 433 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0735899 s, 712 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0497399 s, 1.1 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0663057 s, 791 MB/s
52428800 bytes (52 MB) copied, 0.113188 s, 463 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111403 s, 471 MB/s
52428800 bytes (52 MB) copied, 0.0764811 s, 686 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0435216 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.03914 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0965657 s, 543 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0887902 s, 590 MB/s
52428800 bytes (52 MB) copied, 0.122325 s, 429 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.156659 s, 335 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.187956 s, 279 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142848 s, 367 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122888 s, 427 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136094 s, 385 MB/s
52428800 bytes (52 MB) copied, 0.116564 s, 450 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0804488 s, 652 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122198 s, 429 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106328 s, 493 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119413 s, 439 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153738 s, 341 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0654719 s, 801 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0743756 s, 705 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.163418 s, 321 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.172134 s, 305 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0804576 s, 652 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.030413 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.169675 s, 309 MB/s
, 0.0837886 s, 626 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0699874 s, 749 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0598477 s, 876 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.164865 s, 318 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0355918 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.029574 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258757 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.026435 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0339928 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0517069 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0200906 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0520276 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0623374 s, 841 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0263933 s, 2.0 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0433456 s, 1.2 GB/s
52428800 bytes (52 MB) copied, 0.0692294 s, 757 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0499942 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0331175 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0605841 s, 865 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0948455 s, 553 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0648268 s, 809 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0452353 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0616476 s, 850 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0413551 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.063797 s, 822 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0483461 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0410245 s, 1.3 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0427275 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0894822 s, 586 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.109248 s, 480 MB/s
, 0.0767666 s, 683 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11113 s, 472 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0716128 s, 732 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.114428 s, 458 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0850844 s, 616 MB/s
52428800 bytes (52 MB) copied, 0.0722866 s, 725 MB/s
52428800 bytes (52 MB) copied, 0.0688263 s, 762 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.118343 s, 443 MB/s
52428800 bytes (52 MB) copied, 0.121343 s, 432 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.121174 s, 433 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117068 s, 448 MB/s
, 0.135425 s, 387 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.119073 s, 440 MB/s
, 0.168555 s, 311 MB/s
52428800 bytes (52 MB) copied, 0.123011 s, 426 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112845 s, 465 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10405 s, 504 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127759 s, 410 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106042 s, 494 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151177 s, 347 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.118279 s, 443 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0818657 s, 640 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11457 s, 458 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145591 s, 360 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149189 s, 351 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108437 s, 483 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0691559 s, 758 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0238701 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0423962 s, 1.2 GB/s
52428800 bytes (52 MB) copied, 0.159825 s, 328 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12142 s, 432 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0914851 s, 573 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0257307 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0371984 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.031362 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0256796 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0400338 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0562279 s, 932 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0279915 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0268845 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0328401 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0323419 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.029077 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0304633 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0681747 s, 769 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.045875 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0574897 s, 912 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.09538 s, 550 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0996613 s, 526 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.123961 s, 423 MB/s
52428800 bytes (52 MB) copied, 0.11367 s, 461 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.130625 s, 401 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.1098 s, 477 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.159216 s, 329 MB/s
52428800 bytes (52 MB) copied, 0.145739 s, 360 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.201578 s, 260 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.169546 s, 309 MB/s
52428800 bytes (52 MB) copied, 0.194434 s, 270 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.1676 s, 313 MB/s
52428800 bytes (52 MB) copied, 0.11891 s, 441 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.146147 s, 359 MB/s
, 0.176067 s, 298 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.246556 s, 213 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.204921 s, 256 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.134416 s, 390 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126027 s, 416 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151825 s, 345 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119645 s, 438 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157696 s, 332 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.123774 s, 424 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0843743 s, 621 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0587808 s, 892 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153965 s, 341 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0615772 s, 851 MB/s
, 0.10468 s, 501 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115876 s, 452 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131621 s, 398 MB/s
52428800 bytes (52 MB) copied, 0.145623 s, 360 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.056015 s, 936 MB/s
52428800 bytes (52 MB) copied, 0.0599702 s, 874 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.051126 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0422335 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0341061 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0289057 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0511838 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0266421 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0437399 s, 1.2 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0750251 s, 699 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0561141 s, 934 MB/s
52428800 bytes (52 MB) copied, 0.066383 s, 790 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.05932 s, 884 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0492168 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0653506 s, 802 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0765889 s, 685 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115398 s, 454 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0762449 s, 688 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.088141 s, 595 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0380908 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0639356 s, 820 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0821902 s, 638 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0260222 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0723052 s, 725 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0568618 s, 922 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0229727 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0253116 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0239797 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0243182 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0260413 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0197223 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0411565 s, 1.3 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0195463 s, 2.7 GB/s
52428800 bytes (52 MB) copied, 0.0350405 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0582517 s, 900 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0328081 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0382852 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0349708 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0286147 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0241688 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0365872 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0316912 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0403539 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0528153 s, 993 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0480232 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0956143 s, 548 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0614191 s, 854 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0742604 s, 706 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0820268 s, 639 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0826761 s, 634 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0706587 s, 742 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0651351 s, 805 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0566461 s, 926 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0500215 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0948918 s, 553 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103547 s, 506 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.129502 s, 405 MB/s
52428800 bytes (52 MB) copied, 0.03601 s, 1.5 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.114828 s, 457 MB/s
, 0.0561393 s, 934 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0672144 s, 780 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0589393 s, 890 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.0507367 s, 1.0 GB/s
52428800 bytes (52 MB) copied, 0.0495688 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0310126 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0274317 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.047544 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0259251 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0350835 s, 1.5 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0279417 s, 1.9 GB/s
, 0.0449446 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0299533 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.024056 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0283502 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0245714 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0233649 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0205318 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0301362 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0338324 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.036394 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0342904 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0230696 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0434646 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308379 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0229241 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0270058 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0273381 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0301716 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0329959 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306064 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.042561 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0406125 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0656329 s, 799 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0674026 s, 778 MB/s
52428800 bytes (52 MB) copied, 0.0590161 s, 888 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100644 s, 521 MB/s
52428800 bytes (52 MB) copied, 0.0827622 s, 633 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0672842 s, 779 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0804369 s, 652 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0658161 s, 797 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0615406 s, 852 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107967 s, 486 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0819 s, 640 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0499834 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.07828 s, 670 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.075677 s, 693 MB/s
52428800 bytes (52 MB) copied, 0.0606903 s, 864 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.057135 s, 918 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0198885 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0406163 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0347979 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0232302 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376029 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0227353 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0269977 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0671799 s, 780 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0398992 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0276249 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0448521 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.01945 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.040708 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0424386 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0181239 s, 2.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0912238 s, 575 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0338605 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0342229 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0337616 s, 1.6 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488547 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0439415 s, 1.2 GB/s
52428800 bytes (52 MB) copied, 0.0426561 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0499089 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0550767 s, 952 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0703748 s, 745 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0833107 s, 629 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0840219 s, 624 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110047 s, 476 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0983943 s, 533 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116541 s, 450 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0943578 s, 556 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108348 s, 484 MB/s
52428800 bytes (52 MB) copied, 0.0828011 s, 633 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.150259 s, 349 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0796418 s, 658 MB/s
52428800 bytes (52 MB) copied, 0.0747461 s, 701 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12693 s, 413 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.16012 s, 327 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115128 s, 455 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.135697 s, 386 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0952234 s, 551 MB/s
52428800 bytes (52 MB) copied, 0.0534993 s, 980 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108369 s, 484 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0619531 s, 846 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0366061 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0437178 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0297719 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0285613 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0363805 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0481322 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0246894 s, 2.1 GB/s
, 0.0334272 s, 1.6 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0320015 s, 1.6 GB/s
52428800 bytes (52 MB) copied, 0.0353126 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.042281 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0292293 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0531965 s, 986 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.032937 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0425015 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.027554 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0379588 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0234643 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0312903 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0297097 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0329138 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.036936 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0228155 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0371424 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0355576 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.021053 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0263101 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0286511 s, 1.8 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488085 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0365115 s, 1.4 GB/s
52428800 bytes (52 MB) copied, 0.0468289 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0244883 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0347103 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0553832 s, 947 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0607098 s, 864 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0433503 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0575552 s, 911 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0390218 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0256629 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0442423 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0802905 s, 653 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0638529 s, 821 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0778709 s, 673 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0591512 s, 886 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0419995 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0380702 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0217372 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0339888 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0240744 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0218838 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0251117 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0339936 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0338728 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0242036 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0207943 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0351303 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0290785 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0173631 s, 3.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0234463 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0231591 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0204353 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0253478 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0263489 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0222686 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0206488 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0391842 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0438704 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0241424 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0288702 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0591092 s, 887 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468849 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.0339851 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0973178 s, 539 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0738576 s, 710 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0860177 s, 610 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107449 s, 488 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113833 s, 461 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100326 s, 523 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102509 s, 511 MB/s
52428800 bytes (52 MB) copied, 0.0988185 s, 531 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0906519 s, 578 MB/s
, 0.0972265 s, 539 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113693 s, 461 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0861182 s, 609 MB/s
52428800 bytes (52 MB) copied, 0.102237 s, 513 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126672 s, 414 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103351 s, 507 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0679748 s, 771 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0761097 s, 689 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0938882 s, 558 MB/s
52428800 bytes (52 MB) copied, 0.0764178 s, 686 MB/s
52428800 bytes (52 MB) copied, 0.105462 s, 497 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0300224 s, 1.7 GB/s
52428800 bytes (52 MB) copied, 0.0222089 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0702415 s, 746 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0503348 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0482144 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0733149 s, 715 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0315441 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0272656 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.019254 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0231305 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0278375 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0212839 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0273156 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0215888 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0266596 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0280343 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.020882 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0389571 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0327033 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522086 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0395803 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0356403 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469297 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0430298 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0476421 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0667096 s, 786 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0466334 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0720418 s, 728 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0609468 s, 860 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0621454 s, 844 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0611116 s, 858 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0949884 s, 552 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108648 s, 483 MB/s
52428800 bytes (52 MB) copied, 0.114586 s, 458 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0421754 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0660933 s, 793 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0785099 s, 668 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0450977 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0692024 s, 758 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0291207 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0742781 s, 706 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0378032 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0574521 s, 913 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0377325 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0412573 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0229054 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0378912 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0295184 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287211 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0556868 s, 941 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0542787 s, 966 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0267297 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0334119 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.024651 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0237569 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0674669 s, 777 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.025818 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0251511 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0266866 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0267169 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.027243 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0257951 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0380608 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.046192 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0351628 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0266563 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0260489 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0270013 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0280866 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0262161 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0219548 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.021544 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258525 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0227643 s, 2.3 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352179 s, 1.5 GB/s
52428800 bytes (52 MB) copied, 0.0599413 s, 875 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0372224 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0392063 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0648357 s, 809 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0500226 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.043019 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0537465 s, 975 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0482612 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0553244 s, 948 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0503961 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0876366 s, 598 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0771251 s, 680 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.063318 s, 828 MB/s
52428800 bytes (52 MB) copied, 0.0481149 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0945385 s, 555 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287015 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0654278 s, 801 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.048251 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.0637593 s, 822 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0788087 s, 665 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0614166 s, 854 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0624823 s, 839 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0511382 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.053274 s, 984 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0758239 s, 691 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0772282 s, 679 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0636447 s, 824 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258101 s, 2.0 GB/s
52428800 bytes (52 MB) copied, 0.0519837 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122507 s, 428 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122501 s, 428 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0817006 s, 642 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0732312 s, 716 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0551956 s, 950 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0330113 s, 1.6 GB/s
52428800 bytes (52 MB) copied, 0.0538549 s, 974 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0895071 s, 586 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360342 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0407365 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.020971 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0303817 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0297888 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258642 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0237808 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0275533 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0231201 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0408306 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0367887 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0410487 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0713775 s, 735 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0418002 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0601845 s, 871 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0588043 s, 892 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0443335 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0293441 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0211999 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0545839 s, 961 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0446252 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0305401 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0440381 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0862148 s, 608 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.062677 s, 836 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103414 s, 507 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149366 s, 351 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0622686 s, 842 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.080581 s, 651 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0968894 s, 541 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.150639 s, 348 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.114528 s, 458 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0968621 s, 541 MB/s
52428800 bytes (52 MB) copied, 0.158891 s, 330 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.1043 s, 503 MB/s
, 0.157363 s, 333 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0862807 s, 608 MB/s
52428800 bytes (52 MB) copied, 0.0715714 s, 733 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0739519 s, 709 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0954035 s, 550 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0913501 s, 574 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.140749 s, 372 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.166057 s, 316 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.079731 s, 658 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.263521 s, 199 MB/s
50+0 records in
50+0 records out
, 0.125431 s, 418 MB/s
52428800 bytes (52 MB) copied, 0.222635 s, 235 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0885135 s, 592 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.197636 s, 265 MB/s
52428800 bytes (52 MB) copied, 0.238518 s, 220 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117935 s, 445 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0764748 s, 686 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0467112 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0710705 s, 738 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0671481 s, 781 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0553893 s, 947 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0676103 s, 775 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0775857 s, 676 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0356062 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0227891 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0288289 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0188178 s, 2.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0216621 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.035902 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0373547 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0591437 s, 886 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0615484 s, 852 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.080072 s, 655 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101218 s, 518 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0622582 s, 842 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0895103 s, 586 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102648 s, 511 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0729859 s, 718 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0736561 s, 712 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.065277 s, 803 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0459537 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.106113 s, 494 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0870267 s, 602 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112327 s, 467 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116275 s, 451 MB/s
52428800 bytes (52 MB) copied, 0.180329 s, 291 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113016 s, 464 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.160749 s, 326 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142286 s, 368 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119784 s, 438 MB/s
52428800 bytes (52 MB) copied, 0.0982937 s, 533 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121315 s, 432 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0932969 s, 562 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0866189 s, 605 MB/s
52428800 bytes (52 MB) copied, 0.132001 s, 397 MB/s
52428800 bytes (52 MB) copied, 0.136437 s, 384 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115176 s, 455 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0757725 s, 692 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0736997 s, 711 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0318201 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0988848 s, 530 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.134534 s, 390 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.096427 s, 544 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0887167 s, 591 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.125494 s, 418 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.082748 s, 634 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119064 s, 440 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.0363325 s, 1.4 GB/s
52428800 bytes (52 MB) copied, 0.0883567 s, 593 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0325839 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0251462 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0378168 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0574111 s, 913 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0912954 s, 574 MB/s
52428800 bytes (52 MB) copied, 0.0486923 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0834292 s, 628 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11519 s, 455 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101417 s, 517 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122743 s, 427 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11046 s, 475 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122923 s, 427 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120026 s, 437 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137964 s, 380 MB/s
52428800 bytes (52 MB) copied, 0.115564 s, 454 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138775 s, 378 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.140648 s, 373 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137083 s, 382 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122204 s, 429 MB/s
52428800 bytes (52 MB) copied, 0.137065 s, 383 MB/s
52428800 bytes (52 MB) copied, 0.124613 s, 421 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117772 s, 445 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0901983 s, 581 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0658093 s, 797 MB/s
52428800 bytes (52 MB) copied, 0.0635484 s, 825 MB/s
52428800 bytes (52 MB) copied, 0.121896 s, 430 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105845 s, 495 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0247381 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0495008 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0395086 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352258 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.023726 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0225303 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0366917 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0303804 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.040177 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0813607 s, 644 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10991 s, 477 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120346 s, 436 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0965549 s, 543 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119758 s, 438 MB/s
52428800 bytes (52 MB) copied, 0.074513 s, 704 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119143 s, 440 MB/s
52428800 bytes (52 MB) copied, 0.110037 s, 476 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.144407 s, 363 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115513 s, 454 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0386321 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.135314 s, 387 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.032575 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15631 s, 335 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142557 s, 368 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101003 s, 519 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0310997 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0798087 s, 657 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.150206 s, 349 MB/s
, 0.122477 s, 428 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0572885 s, 915 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138765 s, 378 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0765147 s, 685 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0806382 s, 650 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106842 s, 491 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0945678 s, 554 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103489 s, 507 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.134382 s, 390 MB/s
52428800 bytes (52 MB) copied, 0.0943224 s, 556 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.146688 s, 357 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109492 s, 479 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0793381 s, 661 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15477 s, 339 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0771217 s, 680 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0783794 s, 669 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0971124 s, 540 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0802088 s, 654 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0859678 s, 610 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0912809 s, 574 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116231 s, 451 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110974 s, 472 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.211923 s, 247 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.118977 s, 441 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0655218 s, 800 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10964 s, 478 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120593 s, 435 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.281901 s, 186 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.175476 s, 299 MB/s
52428800 bytes (52 MB) copied, 0.0659355 s, 795 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105263 s, 498 MB/s
52428800 bytes (52 MB) copied, 0.125215 s, 419 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.118952 s, 441 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127524 s, 411 MB/s
52428800 bytes (52 MB) copied, 0.123974 s, 423 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153566 s, 341 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.134974 s, 388 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138643 s, 378 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0950497 s, 552 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0874597 s, 599 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.173532 s, 302 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100849 s, 520 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0689631 s, 760 MB/s
52428800 bytes (52 MB) copied, 0.137297 s, 382 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0812725 s, 645 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0856804 s, 612 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0643561 s, 815 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0881833 s, 595 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0495545 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0357372 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108787 s, 482 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102771 s, 510 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110798 s, 473 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0546504 s, 959 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101329 s, 517 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0830509 s, 631 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100214 s, 523 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.061516 s, 852 MB/s
, 0.109284 s, 480 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.088846 s, 590 MB/s
52428800 bytes (52 MB) copied, 0.103111 s, 508 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0743003 s, 706 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.060373 s, 868 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120921 s, 434 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0365576 s, 1.4 GB/s
52428800 bytes (52 MB) copied, 0.135054 s, 388 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0278247 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0683385 s, 767 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.025651 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0259489 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0269042 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0519103 s, 1.0 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0606527 s, 864 MB/s
52428800 bytes (52 MB) copied, 0.0451126 s, 1.2 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0670445 s, 782 MB/s
, 0.0360257 s, 1.5 GB/s
52428800 bytes (52 MB) copied, 0.0681135 s, 770 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0612267 s, 856 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0472372 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0456237 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0237983 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0217669 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0230782 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0341631 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.033433 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0434274 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0703067 s, 746 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0782806 s, 670 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0924378 s, 567 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.071103 s, 737 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0983023 s, 533 MB/s
52428800 bytes (52 MB) copied, 0.0862922 s, 608 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107167 s, 489 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113606 s, 461 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119943 s, 437 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.133907 s, 392 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0845223 s, 620 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108632 s, 483 MB/s
52428800 bytes (52 MB) copied, 0.107356 s, 488 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.162732 s, 322 MB/s
, 0.118656 s, 442 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0559395 s, 937 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0612154 s, 856 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0911416 s, 575 MB/s
52428800 bytes (52 MB) copied, 0.210477 s, 249 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.198094 s, 265 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.208745 s, 251 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116254 s, 451 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142646 s, 368 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151043 s, 347 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127536 s, 411 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101039 s, 519 MB/s
52428800 bytes (52 MB) copied, 0.108539 s, 483 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.216444 s, 242 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15776 s, 332 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127661 s, 411 MB/s
52428800 bytes (52 MB) copied, 0.0933634 s, 562 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0878532 s, 597 MB/s
52428800 bytes (52 MB) copied, 0.260903 s, 201 MB/s
52428800 bytes (52 MB) copied, 0.155729 s, 337 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0984641 s, 532 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0388552 s, 1.3 GB/s
, 0.0926267 s, 566 MB/s
52428800 bytes (52 MB) copied, 0.0692132 s, 757 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0928251 s, 565 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0395241 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0740894 s, 708 MB/s
52428800 bytes (52 MB) copied, 0.0473809 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0374442 s, 1.4 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0576795 s, 909 MB/s
52428800 bytes (52 MB) copied, 0.0572334 s, 916 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0520515 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0510818 s, 1.0 GB/s
, 0.0784166 s, 669 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0558591 s, 939 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0791619 s, 662 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0679391 s, 772 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.060445 s, 867 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.055004 s, 953 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0799995 s, 655 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0753725 s, 696 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0727414 s, 721 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0556825 s, 942 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0770607 s, 680 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0891551 s, 588 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137373 s, 382 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0925577 s, 566 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.091018 s, 576 MB/s
, 0.117198 s, 447 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102658 s, 511 MB/s
52428800 bytes (52 MB) copied, 0.149096 s, 352 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117983 s, 444 MB/s
52428800 bytes (52 MB) copied, 0.122309 s, 429 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0806524 s, 650 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.118656 s, 442 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0760756 s, 689 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13811 s, 380 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0634459 s, 826 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.166115 s, 316 MB/s
52428800 bytes (52 MB) copied, 0.122373 s, 428 MB/s
52428800 bytes (52 MB) copied, 0.11092 s, 473 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0843387 s, 622 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0877838 s, 597 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.357879 s, 146 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.161937 s, 324 MB/s
52428800 bytes (52 MB) copied, 0.161244 s, 325 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.205045 s, 256 MB/s
52428800 bytes (52 MB) copied, 0.411616 s, 127 MB/s
52428800 bytes (52 MB) copied, 0.303166 s, 173 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.43808 s, 120 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.408105 s, 128 MB/s
52428800 bytes (52 MB) copied, 0.232612 s, 225 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.36864 s, 142 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.243354 s, 215 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.406085 s, 129 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.254525 s, 206 MB/s
52428800 bytes (52 MB) copied, 0.326237 s, 161 MB/s
52428800 bytes (52 MB) copied, 0.299651 s, 175 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.278492 s, 188 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.220718 s, 238 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.382659 s, 137 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.224797 s, 233 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.230294 s, 228 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.163917 s, 320 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.238062 s, 220 MB/s
52428800 bytes (52 MB) copied, 0.264301 s, 198 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.217709 s, 241 MB/s
52428800 bytes (52 MB) copied, 0.261069 s, 201 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.225214 s, 233 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122314 s, 429 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0950297 s, 552 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122705 s, 427 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.270597 s, 194 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142076 s, 369 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0930301 s, 564 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117607 s, 446 MB/s
52428800 bytes (52 MB) copied, 0.0506585 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128357 s, 408 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0402106 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.118182 s, 444 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0282564 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0201434 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0392513 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.037082 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0342825 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0783674 s, 669 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0235518 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.021394 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0326763 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0236281 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0872528 s, 601 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0791261 s, 663 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0564463 s, 929 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142515 s, 368 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136486 s, 384 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.233765 s, 224 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.220829 s, 237 MB/s
, 0.246599 s, 213 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.200583 s, 261 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.301827 s, 174 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.217247 s, 241 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.272233 s, 193 MB/s
52428800 bytes (52 MB) copied, 0.163661 s, 320 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.322865 s, 162 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.258726 s, 203 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.135999 s, 386 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.430327 s, 122 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.307098 s, 171 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102473 s, 512 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.333823 s, 157 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.157207 s, 334 MB/s
, 0.473228 s, 111 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.211059 s, 248 MB/s
52428800 bytes (52 MB) copied, 0.377234 s, 139 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.38685 s, 136 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.330507 s, 159 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.260278 s, 201 MB/s
, 0.267349 s, 196 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.255397 s, 205 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.31304 s, 167 MB/s
52428800 bytes (52 MB) copied, 0.0719048 s, 729 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.26269 s, 200 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.192586 s, 272 MB/s
, 0.275051 s, 191 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.311948 s, 168 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.237132 s, 221 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.294761 s, 178 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0618565 s, 848 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0532806 s, 984 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0614032 s, 854 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0846666 s, 619 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0874514 s, 600 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0417739 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360956 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311428 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0414155 s, 1.3 GB/s
, 0.0400507 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0558494 s, 939 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0838722 s, 625 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0610458 s, 859 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.086523 s, 606 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0470556 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0407065 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0435481 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0837505 s, 626 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0473288 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105906 s, 495 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0524748 s, 999 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0301478 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0678155 s, 773 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0675823 s, 776 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0640064 s, 819 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0710067 s, 738 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.20954 s, 250 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0661982 s, 792 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0293735 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.202683 s, 259 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.189162 s, 277 MB/s
52428800 bytes (52 MB) copied, 0.060574 s, 866 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.189483 s, 277 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0354778 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0496256 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0248925 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0477757 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0334025 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0451353 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0233242 s, 2.2 GB/s
, 0.0309869 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0326429 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0285941 s, 1.8 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0637503 s, 822 MB/s
50+0 records in
50+0 records out
, 0.028768 s, 1.8 GB/s
52428800 bytes (52 MB) copied, 0.0564776 s, 928 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0556096 s, 943 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0803209 s, 653 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137598 s, 381 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0690715 s, 759 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0688397 s, 762 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0968156 s, 542 MB/s
52428800 bytes (52 MB) copied, 0.0901992 s, 581 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102349 s, 512 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137534 s, 381 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.130585 s, 401 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0796971 s, 658 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0919631 s, 570 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0799792 s, 656 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0735655 s, 713 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0807485 s, 649 MB/s
52428800 bytes (52 MB) copied, 0.156669 s, 335 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0454445 s, 1.2 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0537497 s, 975 MB/s
52428800 bytes (52 MB) copied, 0.075294 s, 696 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103237 s, 508 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0746439 s, 702 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468736 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0341464 s, 1.5 GB/s
52428800 bytes (52 MB) copied, 0.0357216 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0426692 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468745 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0329285 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0444851 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0500588 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0448247 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370661 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0457457 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0211999 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0221266 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0431895 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0544993 s, 962 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0268429 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0313887 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0369001 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0236217 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0351357 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0328583 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0256471 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0292683 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336545 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0259657 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0506256 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0514794 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0446901 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0494905 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0641643 s, 817 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0650532 s, 806 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0857081 s, 612 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0476393 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0661526 s, 793 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104739 s, 501 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0385952 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0919754 s, 570 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0739287 s, 709 MB/s
52428800 bytes (52 MB) copied, 0.143429 s, 366 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.158708 s, 330 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.124207 s, 422 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128951 s, 407 MB/s
52428800 bytes (52 MB) copied, 0.0704786 s, 744 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0940714 s, 557 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11479 s, 457 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.045854 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.0948788 s, 553 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.061964 s, 846 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0897514 s, 584 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0715444 s, 733 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0658 s, 797 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.064574 s, 812 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0403977 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0390521 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0518244 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.044057 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0482536 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100124 s, 524 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0740579 s, 708 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.134246 s, 391 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137293 s, 382 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.156392 s, 335 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.148593 s, 353 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.161018 s, 326 MB/s
52428800 bytes (52 MB) copied, 0.142615 s, 368 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.161606 s, 324 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.210095 s, 250 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.164611 s, 319 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.296008 s, 177 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.148081 s, 354 MB/s
52428800 bytes (52 MB) copied, 0.261665 s, 200 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149239 s, 351 MB/s
, 0.350851 s, 149 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.202953 s, 258 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126575 s, 414 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0758499 s, 691 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.129114 s, 406 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.275119 s, 191 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157325 s, 333 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.255323 s, 205 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.321682 s, 163 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.385577 s, 136 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.14926 s, 351 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.143941 s, 364 MB/s
52428800 bytes (52 MB) copied, 0.148552 s, 353 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.166595 s, 315 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0702833 s, 746 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.199307 s, 263 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15998 s, 328 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103452 s, 507 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113227 s, 463 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0532582 s, 984 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0699393 s, 750 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0320742 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0206167 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0249608 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.049582 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0340948 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0315111 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0509694 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0337099 s, 1.6 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.042058 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0317305 s, 1.7 GB/s
52428800 bytes (52 MB) copied, 0.0810623 s, 647 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0345125 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0651551 s, 805 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0676744 s, 775 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0762726 s, 687 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0815199 s, 643 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0223144 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0647911 s, 809 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0634048 s, 827 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0614143 s, 854 MB/s
52428800 bytes (52 MB) copied, 0.0449227 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.125065 s, 419 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.044629 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0662047 s, 792 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0664966 s, 788 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0383109 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0328964 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138387 s, 379 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0226461 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102696 s, 511 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0244036 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0731006 s, 717 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0302652 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0228475 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0318413 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360176 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306217 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0398044 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.03151 s, 1.7 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101598 s, 516 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104392 s, 502 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0645067 s, 813 MB/s
52428800 bytes (52 MB) copied, 0.0924259 s, 567 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.079251 s, 662 MB/s
, 0.0919265 s, 570 MB/s
52428800 bytes (52 MB) copied, 0.107518 s, 488 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124287 s, 422 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.199044 s, 263 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151474 s, 346 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.236044 s, 222 MB/s
, 0.222337 s, 236 MB/s
52428800 bytes (52 MB) copied, 0.17208 s, 305 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.234823 s, 223 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.198216 s, 265 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.262979 s, 199 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.347369 s, 151 MB/s
50+0 records in
50+0 records out
, 0.249287 s, 210 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.194074 s, 270 MB/s
52428800 bytes (52 MB) copied, 0.222535 s, 236 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.264396 s, 198 MB/s
52428800 bytes (52 MB) copied, 0.140659 s, 373 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.32131 s, 163 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.315541 s, 166 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.264434 s, 198 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.298149 s, 176 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.461697 s, 114 MB/s
52428800 bytes (52 MB) copied, 0.313994 s, 167 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.225145 s, 233 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.378241 s, 139 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.26949 s, 195 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.156123 s, 336 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.306265 s, 171 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0414476 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.263884 s, 199 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.337401 s, 155 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.310998 s, 169 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0369807 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.315001 s, 166 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0377531 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.178793 s, 293 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.20586 s, 255 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115495 s, 454 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360382 s, 1.5 GB/s
52428800 bytes (52 MB) copied, 0.124869 s, 420 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336853 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0415265 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119522 s, 439 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10925 s, 480 MB/s
52428800 bytes (52 MB) copied, 0.0914456 s, 573 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258381 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0545603 s, 961 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0885841 s, 592 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.037868 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0324197 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0328156 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522777 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0421459 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0444869 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0232396 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0407013 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0246035 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0356066 s, 1.5 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.045325 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0365058 s, 1.4 GB/s
52428800 bytes (52 MB) copied, 0.0418693 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0685802 s, 764 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0697285 s, 752 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0928991 s, 564 MB/s
52428800 bytes (52 MB) copied, 0.113372 s, 462 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0836547 s, 627 MB/s
52428800 bytes (52 MB) copied, 0.103329 s, 507 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0952137 s, 551 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0981486 s, 534 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.134795 s, 389 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119126 s, 440 MB/s
52428800 bytes (52 MB) copied, 0.0843635 s, 621 MB/s
52428800 bytes (52 MB) copied, 0.0676727 s, 775 MB/s
52428800 bytes (52 MB) copied, 0.0861465 s, 609 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105381 s, 498 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0999778 s, 524 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0540442 s, 970 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0433392 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0708223 s, 740 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103203 s, 508 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0475956 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0871679 s, 601 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0647927 s, 809 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0909735 s, 576 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0952473 s, 550 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.135015 s, 388 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0875946 s, 599 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0948806 s, 553 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122453 s, 428 MB/s
52428800 bytes (52 MB) copied, 0.111659 s, 470 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107784 s, 486 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106958 s, 490 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100093 s, 524 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0958976 s, 547 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157855 s, 332 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15162 s, 346 MB/s
, 0.157376 s, 333 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121966 s, 430 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.133591 s, 392 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0668026 s, 785 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0405727 s, 1.3 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131226 s, 400 MB/s
52428800 bytes (52 MB) copied, 0.13259 s, 395 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.241439 s, 217 MB/s
, 0.157377 s, 333 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.114422 s, 458 MB/s
52428800 bytes (52 MB) copied, 0.11522 s, 455 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.133524 s, 393 MB/s
52428800 bytes (52 MB) copied, 0.172667 s, 304 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128066 s, 409 MB/s
52428800 bytes (52 MB) copied, 0.154913 s, 338 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117294 s, 447 MB/s
52428800 bytes (52 MB) copied, 0.135824 s, 386 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13996 s, 375 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.183562 s, 286 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.139762 s, 375 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.165444 s, 317 MB/s
52428800 bytes (52 MB) copied, 0.0993222 s, 528 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.146475 s, 358 MB/s
52428800 bytes (52 MB) copied, 0.0753013 s, 696 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0357501 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0350053 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.139229 s, 377 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153122 s, 342 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.232099 s, 226 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.226941 s, 231 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.16461 s, 319 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120421 s, 435 MB/s
52428800 bytes (52 MB) copied, 0.195024 s, 269 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.202057 s, 259 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0489486 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0345444 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0269618 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0304927 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0381447 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0497435 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0273852 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0799122 s, 656 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0812709 s, 645 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0671098 s, 781 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.092784 s, 565 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0790504 s, 663 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0599185 s, 875 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0767893 s, 683 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103339 s, 507 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137445 s, 381 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0879326 s, 596 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0606156 s, 865 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151893 s, 345 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0500614 s, 1.0 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149195 s, 351 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0888787 s, 590 MB/s
52428800 bytes (52 MB) copied, 0.104075 s, 504 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0904738 s, 579 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.074911 s, 700 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0444451 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0862846 s, 608 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0872552 s, 601 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0813705 s, 644 MB/s
52428800 bytes (52 MB) copied, 0.0686052 s, 764 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.176416 s, 297 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.170302 s, 308 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157753 s, 332 MB/s
52428800 bytes (52 MB) copied, 0.129127 s, 406 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.216191 s, 243 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.154703 s, 339 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11131 s, 471 MB/s
52428800 bytes (52 MB) copied, 0.170095 s, 308 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.070689 s, 742 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.176982 s, 296 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0942442 s, 556 MB/s
, 0.113695 s, 461 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.256334 s, 205 MB/s
52428800 bytes (52 MB) copied, 0.0860854 s, 609 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0634277 s, 827 MB/s
52428800 bytes (52 MB) copied, 0.164376 s, 319 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0858426 s, 611 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.176445 s, 297 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.284983 s, 184 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0765403 s, 685 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.146926 s, 357 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.271442 s, 193 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.160299 s, 327 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.29525 s, 178 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.226479 s, 231 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108172 s, 485 MB/s
52428800 bytes (52 MB) copied, 0.0693983 s, 755 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127613 s, 411 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121849 s, 430 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0906955 s, 578 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.159193 s, 329 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113728 s, 461 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0744006 s, 705 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.231557 s, 226 MB/s
52428800 bytes (52 MB) copied, 0.18205 s, 288 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105083 s, 499 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.167868 s, 312 MB/s
52428800 bytes (52 MB) copied, 0.164224 s, 319 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.178626 s, 294 MB/s
52428800 bytes (52 MB) copied, 0.169614 s, 309 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.095648 s, 548 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.143825 s, 365 MB/s
52428800 bytes (52 MB) copied, 0.0774273 s, 677 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.260663 s, 201 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.163326 s, 321 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.165758 s, 316 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136641 s, 384 MB/s
52428800 bytes (52 MB) copied, 0.160878 s, 326 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137954 s, 380 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.282227 s, 186 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.175743 s, 298 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.249113 s, 210 MB/s
52428800 bytes (52 MB) copied, 0.131859 s, 398 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119576 s, 438 MB/s
52428800 bytes (52 MB) copied, 0.261626 s, 200 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0937134 s, 559 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0978961 s, 536 MB/s
52428800 bytes (52 MB) copied, 0.135333 s, 387 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.256656 s, 204 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12608 s, 416 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13413 s, 391 MB/s
52428800 bytes (52 MB) copied, 0.162319 s, 323 MB/s
52428800 bytes (52 MB) copied, 0.297369 s, 176 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109057 s, 481 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.242692 s, 216 MB/s
52428800 bytes (52 MB) copied, 0.218087 s, 240 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149531 s, 351 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136979 s, 383 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.262705 s, 200 MB/s
52428800 bytes (52 MB) copied, 0.208197 s, 252 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.22534 s, 233 MB/s
52428800 bytes (52 MB) copied, 0.146732 s, 357 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.23848 s, 220 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.191372 s, 274 MB/s
52428800 bytes (52 MB) copied, 0.136141 s, 385 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.188374 s, 278 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.197498 s, 265 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116529 s, 450 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.188826 s, 278 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.141928 s, 369 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12171 s, 431 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0960082 s, 546 MB/s
52428800 bytes (52 MB) copied, 0.195593 s, 268 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.176546 s, 297 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0400641 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157743 s, 332 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0645281 s, 812 MB/s
52428800 bytes (52 MB) copied, 0.0907728 s, 578 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0222201 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0408764 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0536997 s, 976 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0272415 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.221525 s, 237 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0923783 s, 568 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0918443 s, 571 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0727336 s, 721 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0645735 s, 812 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0764505 s, 686 MB/s
52428800 bytes (52 MB) copied, 0.0834361 s, 628 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111329 s, 471 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.140214 s, 374 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.166805 s, 314 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126311 s, 415 MB/s
52428800 bytes (52 MB) copied, 0.144381 s, 363 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.177365 s, 296 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.252083 s, 208 MB/s
52428800 bytes (52 MB) copied, 0.196018 s, 267 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.286913 s, 183 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.195695 s, 268 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.234428 s, 224 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.281227 s, 186 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.260591 s, 201 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.366191 s, 143 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.411621 s, 127 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.329593 s, 159 MB/s
52428800 bytes (52 MB) copied, 0.400773 s, 131 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.264323 s, 198 MB/s
52428800 bytes (52 MB) copied, 0.333226 s, 157 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.454735 s, 115 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.385756 s, 136 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.28846 s, 182 MB/s
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.275632 s, 190 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.322486 s, 163 MB/s
, 0.465384 s, 113 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.218728 s, 240 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.317531 s, 165 MB/s
, 0.342938 s, 153 MB/s
52428800 bytes (52 MB) copied, 0.333037 s, 157 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.238663 s, 220 MB/s
52428800 bytes (52 MB) copied, 0.332023 s, 158 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.205994 s, 255 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.299417 s, 175 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.174651 s, 300 MB/s
52428800 bytes (52 MB) copied, 0.190001 s, 276 MB/s
52428800 bytes (52 MB) copied, 0.197971 s, 265 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.156885 s, 334 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0999745 s, 524 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0509035 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.268533 s, 195 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.05149 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0707063 s, 742 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0368715 s, 1.4 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0434909 s, 1.2 GB/s
52428800 bytes (52 MB) copied, 0.0394946 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0494034 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0508021 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.139114 s, 377 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.139415 s, 376 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.129293 s, 406 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.158728 s, 330 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.159956 s, 328 MB/s
52428800 bytes (52 MB) copied, 0.140262 s, 374 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.141366 s, 371 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.180019 s, 291 MB/s
, 0.179129 s, 293 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.171568 s, 306 MB/s
52428800 bytes (52 MB) copied, 0.17978 s, 292 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.174155 s, 301 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.168066 s, 312 MB/s
52428800 bytes (52 MB) copied, 0.164709 s, 318 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.146942 s, 357 MB/s
, 0.228633 s, 229 MB/s
52428800 bytes (52 MB) copied, 0.120952 s, 433 MB/s
, 0.195447 s, 268 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0865937 s, 605 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12091 s, 434 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.141081 s, 372 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.203291 s, 258 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0731723 s, 717 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.192846 s, 272 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.180942 s, 290 MB/s
52428800 bytes (52 MB) copied, 0.204505 s, 256 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.135513 s, 387 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111661 s, 470 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120792 s, 434 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.186289 s, 281 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.14379 s, 365 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0374627 s, 1.4 GB/s
52428800 bytes (52 MB) copied, 0.153761 s, 341 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.165462 s, 317 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.161616 s, 324 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0399964 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0746896 s, 702 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0476016 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.154275 s, 340 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121949 s, 430 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0727625 s, 721 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.158593 s, 331 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.129486 s, 405 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104679 s, 501 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.114776 s, 457 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121285 s, 432 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151463 s, 346 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.148267 s, 354 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0541825 s, 968 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.156102 s, 336 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13172 s, 398 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.154462 s, 339 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.194277 s, 270 MB/s
52428800 bytes (52 MB) copied, 0.218074 s, 240 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.162495 s, 323 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.182825 s, 287 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.178989 s, 293 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.190851 s, 275 MB/s
52428800 bytes (52 MB) copied, 0.156728 s, 335 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119407 s, 439 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109313 s, 480 MB/s
52428800 bytes (52 MB) copied, 0.206612 s, 254 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.227161 s, 231 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.176613 s, 297 MB/s
52428800 bytes (52 MB) copied, 0.148894 s, 352 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.200911 s, 261 MB/s
52428800 bytes (52 MB) copied, 0.172537 s, 304 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.1754 s, 299 MB/s
52428800 bytes (52 MB) copied, 0.153102 s, 342 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.160691 s, 326 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12207 s, 429 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131598 s, 398 MB/s
52428800 bytes (52 MB) copied, 0.273518 s, 192 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.18154 s, 289 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117328 s, 447 MB/s
52428800 bytes (52 MB) copied, 0.149053 s, 352 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13336 s, 393 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.201351 s, 260 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.172033 s, 305 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0931674 s, 563 MB/s
52428800 bytes (52 MB) copied, 0.165914 s, 316 MB/s
52428800 bytes (52 MB) copied, 0.146794 s, 357 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0846532 s, 619 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.155853 s, 336 MB/s
52428800 bytes (52 MB) copied, 0.0875522 s, 599 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128519 s, 408 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112691 s, 465 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.134252 s, 391 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12923 s, 406 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0774 s, 677 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.160989 s, 326 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0492876 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.16348 s, 321 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.147163 s, 356 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112141 s, 468 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108695 s, 482 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127209 s, 412 MB/s
52428800 bytes (52 MB) copied, 0.241458 s, 217 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111614 s, 470 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.133819 s, 392 MB/s
52428800 bytes (52 MB) copied, 0.188605 s, 278 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.174775 s, 300 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.238266 s, 220 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.174514 s, 300 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.170173 s, 308 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.155846 s, 336 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145869 s, 359 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.213194 s, 246 MB/s
52428800 bytes (52 MB) copied, 0.237911 s, 220 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131286 s, 399 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149666 s, 350 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10557 s, 497 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117245 s, 447 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127756 s, 410 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104244 s, 503 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.25098 s, 209 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.364493 s, 144 MB/s
52428800 bytes (52 MB) copied, 0.129297 s, 405 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.188547 s, 278 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.139753 s, 375 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.189042 s, 277 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.218092 s, 240 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11477 s, 457 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.18215 s, 288 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.226625 s, 231 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151216 s, 347 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136935 s, 383 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.187231 s, 280 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.164684 s, 318 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.139645 s, 375 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122862 s, 427 MB/s
52428800 bytes (52 MB) copied, 0.221812 s, 236 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.163612 s, 320 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.182698 s, 287 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110748 s, 473 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.156735 s, 335 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.183714 s, 285 MB/s
52428800 bytes (52 MB) copied, 0.0698159 s, 751 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.132295 s, 396 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.133834 s, 392 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.256428 s, 204 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116996 s, 448 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0548017 s, 957 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0722357 s, 726 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103055 s, 509 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.0851121 s, 616 MB/s
52428800 bytes (52 MB) copied, 0.140479 s, 373 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.260748 s, 201 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110578 s, 474 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0766664 s, 684 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0760321 s, 690 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.139922 s, 375 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0935816 s, 560 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.176923 s, 296 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109661 s, 478 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101563 s, 516 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.186151 s, 282 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.218347 s, 240 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142676 s, 367 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.214636 s, 244 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.199342 s, 263 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111871 s, 469 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110206 s, 476 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.154836 s, 339 MB/s
52428800 bytes (52 MB) copied, 0.122434 s, 428 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.187301 s, 280 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.165846 s, 316 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
50+0 records in
50+0 records out
, 0.101134 s, 518 MB/s
52428800 bytes (52 MB) copied, 0.103847 s, 505 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.159703 s, 328 MB/s
52428800 bytes (52 MB) copied, 0.135003 s, 388 MB/s
52428800 bytes (52 MB) copied, 0.152903 s, 343 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0962768 s, 545 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.160991 s, 326 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117924 s, 445 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.141517 s, 370 MB/s
52428800 bytes (52 MB) copied, 0.111659 s, 470 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115424 s, 454 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0731732 s, 717 MB/s
, 0.104329 s, 503 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15052 s, 348 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0319015 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0558933 s, 938 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0409422 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.078755 s, 666 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0413692 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0748834 s, 700 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.123193 s, 426 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.057843 s, 906 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.082998 s, 632 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0581519 s, 902 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.180172 s, 291 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115653 s, 453 MB/s
, 0.146264 s, 358 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.174069 s, 301 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157533 s, 333 MB/s
52428800 bytes (52 MB) copied, 0.16078 s, 326 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.177628 s, 295 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.217257 s, 241 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.23167 s, 226 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.254805 s, 206 MB/s
52428800 bytes (52 MB) copied, 0.254799 s, 206 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.192259 s, 273 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.225371 s, 233 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.223864 s, 234 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.226323 s, 232 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.14768 s, 355 MB/s
, 0.226971 s, 231 MB/s
52428800 bytes (52 MB) copied, 0.237224 s, 221 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.347868 s, 151 MB/s
52428800 bytes (52 MB) copied, 0.29588 s, 177 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.148158 s, 354 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.264009 s, 199 MB/s
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.285685 s, 184 MB/s
, 0.272603 s, 192 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111396 s, 471 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0865338 s, 606 MB/s
52428800 bytes (52 MB) copied, 0.0974296 s, 538 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.322667 s, 162 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.160294 s, 327 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.325643 s, 161 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.138268 s, 379 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.273319 s, 192 MB/s
52428800 bytes (52 MB) copied, 0.279448 s, 188 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.064138 s, 817 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0869841 s, 603 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.249669 s, 210 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0246503 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0879328 s, 596 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0783648 s, 669 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0337016 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0367288 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0419046 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0231913 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0327478 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0368683 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0354661 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.032461 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0271951 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0702735 s, 746 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10873 s, 482 MB/s
52428800 bytes (52 MB) copied, 0.110021 s, 477 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127258 s, 412 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.146906 s, 357 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.129578 s, 405 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.184693 s, 284 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.210534 s, 249 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.198191 s, 265 MB/s
52428800 bytes (52 MB) copied, 0.188863 s, 278 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.234308 s, 224 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.26429 s, 198 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.206612 s, 254 MB/s
52428800 bytes (52 MB) copied, 0.177543 s, 295 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.225511 s, 232 MB/s
52428800 bytes (52 MB) copied, 0.345976 s, 152 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.241761 s, 217 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.258608 s, 203 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.244675 s, 214 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.261115 s, 201 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.259598 s, 202 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.185771 s, 282 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.276754 s, 189 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.132809 s, 395 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.17835 s, 294 MB/s
52428800 bytes (52 MB) copied, 0.173371 s, 302 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.170055 s, 308 MB/s
52428800 bytes (52 MB) copied, 0.263975 s, 199 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.332827 s, 158 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.226604 s, 231 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.222639 s, 235 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102555 s, 511 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.310834 s, 169 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.178755 s, 293 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0747856 s, 701 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.226001 s, 232 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0398565 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0511121 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0565995 s, 926 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0839989 s, 624 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0985469 s, 532 MB/s
52428800 bytes (52 MB) copied, 0.102776 s, 510 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108312 s, 484 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145344 s, 361 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.139699 s, 375 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126095 s, 416 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.185227 s, 283 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.197423 s, 266 MB/s
52428800 bytes (52 MB) copied, 0.133847 s, 392 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.181796 s, 288 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.178144 s, 294 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.194273 s, 270 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.171442 s, 306 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.165248 s, 317 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109963 s, 477 MB/s
, 0.153575 s, 341 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.253935 s, 206 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.287345 s, 182 MB/s
52428800 bytes (52 MB) copied, 0.191386 s, 274 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102044 s, 514 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.205853 s, 255 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0677377 s, 774 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.216946 s, 242 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106762 s, 491 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.26319 s, 199 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0293268 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.107706 s, 487 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0939633 s, 558 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.147972 s, 354 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.200963 s, 261 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.237058 s, 221 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.046842 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.175875 s, 298 MB/s
52428800 bytes (52 MB) copied, 0.209907 s, 250 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.168436 s, 311 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0842959 s, 622 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360324 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0500551 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0601819 s, 871 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0656329 s, 799 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.092748 s, 565 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0791586 s, 662 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115678 s, 453 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0735628 s, 713 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.098527 s, 532 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0763065 s, 687 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110804 s, 473 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122546 s, 428 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.16517 s, 317 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127118 s, 412 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.17171 s, 305 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.165948 s, 316 MB/s
52428800 bytes (52 MB) copied, 0.0935695 s, 560 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.158548 s, 331 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.173892 s, 302 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.189402 s, 277 MB/s
52428800 bytes (52 MB) copied, 0.179381 s, 292 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.147678 s, 355 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.135346 s, 387 MB/s
, 0.204811 s, 256 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.23919 s, 219 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.164172 s, 319 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.187692 s, 279 MB/s
52428800 bytes (52 MB) copied, 0.168276 s, 312 MB/s
52428800 bytes (52 MB) copied, 0.133367 s, 393 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128435 s, 408 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.114512 s, 458 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.262743 s, 200 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145322 s, 361 MB/s
52428800 bytes (52 MB) copied, 0.208647 s, 251 MB/s
52428800 bytes (52 MB) copied, 0.348725 s, 150 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.229301 s, 229 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.225087 s, 233 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.206492 s, 254 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.285632 s, 184 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.20017 s, 262 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106704 s, 491 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.194479 s, 270 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.220062 s, 238 MB/s
52428800 bytes (52 MB) copied, 0.175324 s, 299 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.213816 s, 245 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137824 s, 380 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0638041 s, 822 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.261182 s, 201 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0969062 s, 541 MB/s
52428800 bytes (52 MB) copied, 0.063635 s, 824 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0718857 s, 729 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.292798 s, 179 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.344836 s, 152 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126399 s, 415 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.290843 s, 180 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153758 s, 341 MB/s
52428800 bytes (52 MB) copied, 0.0742691 s, 706 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0990732 s, 529 MB/s
52428800 bytes (52 MB) copied, 0.0388408 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127892 s, 410 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101158 s, 518 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0651906 s, 804 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0917353 s, 572 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0988296 s, 530 MB/s
52428800 bytes (52 MB) copied, 0.105812 s, 495 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0963011 s, 544 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.188076 s, 279 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.270173 s, 194 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.181831 s, 288 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.245412 s, 214 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.283353 s, 185 MB/s
52428800 bytes (52 MB) copied, 0.236522 s, 222 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.224319 s, 234 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.327949 s, 160 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.311773 s, 168 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.184334 s, 284 MB/s
52428800 bytes (52 MB) copied, 0.230678 s, 227 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.221168 s, 237 MB/s
52428800 bytes (52 MB) copied, 0.324902 s, 161 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.215541 s, 243 MB/s
52428800 bytes (52 MB) copied, 0.197665 s, 265 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.356211 s, 147 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.304644 s, 172 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111475 s, 470 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.232881 s, 225 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.28925 s, 181 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.199867 s, 262 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.144934 s, 362 MB/s
52428800 bytes (52 MB) copied, 0.268639 s, 195 MB/s
52428800 bytes (52 MB) copied, 0.297842 s, 176 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.254704 s, 206 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.167179 s, 314 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.162659 s, 322 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.155294 s, 338 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.270001 s, 194 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.17531 s, 299 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.248742 s, 211 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.184363 s, 284 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.322483 s, 163 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.137338 s, 382 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.126338 s, 415 MB/s
52428800 bytes (52 MB) copied, 0.12656 s, 414 MB/s
52428800 bytes (52 MB) copied, 0.218553 s, 240 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0905296 s, 579 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.190366 s, 275 MB/s
52428800 bytes (52 MB) copied, 0.13441 s, 390 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0789089 s, 664 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0823522 s, 637 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.121986 s, 430 MB/s
52428800 bytes (52 MB) copied, 0.140576 s, 373 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.182397 s, 287 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.251032 s, 209 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.147021 s, 357 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.152081 s, 345 MB/s
52428800 bytes (52 MB) copied, 0.216456 s, 242 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112841 s, 465 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.206923 s, 253 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.178967 s, 293 MB/s
52428800 bytes (52 MB) copied, 0.151335 s, 346 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120211 s, 436 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0884237 s, 593 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0831612 s, 630 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0789362 s, 664 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.143916 s, 364 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0618128 s, 848 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153779 s, 341 MB/s
52428800 bytes (52 MB) copied, 0.070222 s, 747 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.269225 s, 195 MB/s
52428800 bytes (52 MB) copied, 0.148271 s, 354 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.182276 s, 288 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0848598 s, 618 MB/s
52428800 bytes (52 MB) copied, 0.0941935 s, 557 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.10228 s, 513 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0940428 s, 557 MB/s
52428800 bytes (52 MB) copied, 0.217543 s, 241 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.232789 s, 225 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.216112 s, 243 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.146996 s, 357 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153606 s, 341 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.321003 s, 163 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.224396 s, 234 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.287968 s, 182 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.278384 s, 188 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.298956 s, 175 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.218573 s, 240 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.150168 s, 349 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.272149 s, 193 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117442 s, 446 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.311029 s, 169 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.313287 s, 167 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.309132 s, 170 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.179784 s, 292 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112439 s, 466 MB/s
, 0.391176 s, 134 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.150379 s, 349 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.388228 s, 135 MB/s
52428800 bytes (52 MB) copied, 0.176473 s, 297 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.314889 s, 166 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.494286 s, 106 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.132096 s, 397 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.194718 s, 269 MB/s
52428800 bytes (52 MB) copied, 0.09138 s, 574 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.180053 s, 291 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.201124 s, 261 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.195269 s, 268 MB/s
52428800 bytes (52 MB) copied, 0.0951749 s, 551 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.166819 s, 314 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.148344 s, 353 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111456 s, 470 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.23414 s, 224 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.270687 s, 194 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.221632 s, 237 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.255682 s, 205 MB/s
52428800 bytes (52 MB) copied, 0.273469 s, 192 MB/s
52428800 bytes (52 MB) copied, 0.21869 s, 240 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.265185 s, 198 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.336963 s, 156 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.291882 s, 180 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.282732 s, 185 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.203505 s, 258 MB/s
52428800 bytes (52 MB) copied, 0.272451 s, 192 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.340005 s, 154 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.308818 s, 170 MB/s
52428800 bytes (52 MB) copied, 0.336098 s, 156 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.296402 s, 177 MB/s
52428800 bytes (52 MB) copied, 0.331358 s, 158 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.261876 s, 200 MB/s
52428800 bytes (52 MB) copied, 0.231397 s, 227 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.245969 s, 213 MB/s
, 0.313043 s, 167 MB/s
52428800 bytes (52 MB) copied, 0.243019 s, 216 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.220558 s, 238 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.159419 s, 329 MB/s
52428800 bytes (52 MB) copied, 0.11608 s, 452 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0955981 s, 548 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.243484 s, 215 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.32501 s, 161 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.232542 s, 225 MB/s
52428800 bytes (52 MB) copied, 0.13247 s, 396 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0517574 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.182206 s, 288 MB/s
52428800 bytes (52 MB) copied, 0.0442265 s, 1.2 GB/s
52428800 bytes (52 MB) copied, 0.233767 s, 224 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.148201 s, 354 MB/s
52428800 bytes (52 MB) copied, 0.246082 s, 213 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0859186 s, 610 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136668 s, 384 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.152599 s, 344 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110585 s, 474 MB/s
52428800 bytes (52 MB) copied, 0.110503 s, 474 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.176805 s, 297 MB/s
52428800 bytes (52 MB) copied, 0.167965 s, 312 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157858 s, 332 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.270649 s, 194 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120085 s, 437 MB/s
52428800 bytes (52 MB) copied, 0.143425 s, 366 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.143199 s, 366 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.286293 s, 183 MB/s
52428800 bytes (52 MB) copied, 0.306962 s, 171 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.201952 s, 260 MB/s
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.15789 s, 332 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.173785 s, 302 MB/s
, 0.261451 s, 201 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.424185 s, 124 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.272736 s, 192 MB/s
52428800 bytes (52 MB) copied, 0.219626 s, 239 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.258249 s, 203 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.334357 s, 157 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.30491 s, 172 MB/s
52428800 bytes (52 MB) copied, 0.22831 s, 230 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.254515 s, 206 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.400416 s, 131 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.299774 s, 175 MB/s
52428800 bytes (52 MB) copied, 0.32099 s, 163 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.222296 s, 236 MB/s
52428800 bytes (52 MB) copied, 0.12903 s, 406 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131955 s, 397 MB/s
52428800 bytes (52 MB) copied, 0.177938 s, 295 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0802268 s, 654 MB/s
52428800 bytes (52 MB) copied, 0.193327 s, 271 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.124374 s, 422 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.22037 s, 238 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0642404 s, 816 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.196151 s, 267 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.228197 s, 230 MB/s
52428800 bytes (52 MB) copied, 0.098975 s, 530 MB/s
52428800 bytes (52 MB) copied, 0.261563 s, 200 MB/s
52428800 bytes (52 MB) copied, 0.342537 s, 153 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.247923 s, 211 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0621348 s, 844 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0368634 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.195858 s, 268 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0196726 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.150245 s, 349 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0332947 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0721763 s, 726 MB/s
, 0.104064 s, 504 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0569627 s, 920 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0811013 s, 646 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0348036 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0435448 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0387153 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0960082 s, 546 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.1 s, 524 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0775936 s, 676 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.069264 s, 757 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0589441 s, 889 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0603507 s, 869 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0887451 s, 591 MB/s
52428800 bytes (52 MB) copied, 0.101534 s, 516 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0676549 s, 775 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0941112 s, 557 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110135 s, 476 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0922556 s, 568 MB/s
52428800 bytes (52 MB) copied, 0.117139 s, 448 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105985 s, 495 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120357 s, 436 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0879901 s, 596 MB/s
52428800 bytes (52 MB) copied, 0.133339 s, 393 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110707 s, 474 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0878253 s, 597 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149516 s, 351 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.148588 s, 353 MB/s
52428800 bytes (52 MB) copied, 0.119611 s, 438 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.123248 s, 425 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.143693 s, 365 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136099 s, 385 MB/s
52428800 bytes (52 MB) copied, 0.069107 s, 759 MB/s
52428800 bytes (52 MB) copied, 0.0823791 s, 636 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157461 s, 333 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0986851 s, 531 MB/s
52428800 bytes (52 MB) copied, 0.210254 s, 249 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0912967 s, 574 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145524 s, 360 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.16081 s, 326 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.181564 s, 289 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.195451 s, 268 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.205829 s, 255 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.179456 s, 292 MB/s
52428800 bytes (52 MB) copied, 0.16523 s, 317 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.189049 s, 277 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.178257 s, 294 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12515 s, 419 MB/s
52428800 bytes (52 MB) copied, 0.147347 s, 356 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.106763 s, 491 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0722584 s, 726 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0975764 s, 537 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.05982 s, 876 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308168 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522994 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108316 s, 484 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0745426 s, 703 MB/s
, 0.0690857 s, 759 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0699904 s, 749 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115406 s, 454 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.1149 s, 456 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.157452 s, 333 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.115995 s, 452 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111409 s, 471 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.13963 s, 375 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0913736 s, 574 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.173131 s, 303 MB/s
52428800 bytes (52 MB) copied, 0.17181 s, 305 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112077 s, 468 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120995 s, 433 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.131311 s, 399 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103425 s, 507 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.200195 s, 262 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102753 s, 510 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.154446 s, 339 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.11446 s, 458 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117201 s, 447 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0619855 s, 846 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.072708 s, 721 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.122941 s, 426 MB/s
52428800 bytes (52 MB) copied, 0.173967 s, 301 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0863444 s, 607 MB/s
52428800 bytes (52 MB) copied, 0.188978 s, 277 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.136274 s, 385 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111662 s, 470 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.155811 s, 336 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145457 s, 360 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.163341 s, 321 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.219133 s, 239 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.181189 s, 289 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.264126 s, 198 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.265973 s, 197 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.20863 s, 251 MB/s
52428800 bytes (52 MB) copied, 0.219105 s, 239 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.192003 s, 273 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.389407 s, 135 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.221626 s, 237 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.218977 s, 239 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.172422 s, 304 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.206529 s, 254 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.179229 s, 293 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.339625 s, 154 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.172591 s, 304 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.208555 s, 251 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.20494 s, 256 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.300161 s, 175 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0807997 s, 649 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0999964 s, 524 MB/s
52428800 bytes (52 MB) copied, 0.125065 s, 419 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.177351 s, 296 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0630517 s, 832 MB/s
52428800 bytes (52 MB) copied, 0.121275 s, 432 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.098782 s, 531 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12799 s, 410 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0727081 s, 721 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.200199 s, 262 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0603183 s, 869 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0525027 s, 999 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0261142 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0349424 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.048926 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258921 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0406712 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0379512 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0244083 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0571252 s, 918 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0561416 s, 934 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0408572 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0477521 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0447643 s, 1.2 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.148039 s, 354 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.261059 s, 201 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.213158 s, 246 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.181081 s, 290 MB/s
, 0.104913 s, 500 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.250219 s, 210 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.145509 s, 360 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.128252 s, 409 MB/s
52428800 bytes (52 MB) copied, 0.126268 s, 415 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.173477 s, 302 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0991518 s, 529 MB/s
, 0.236445 s, 222 MB/s
52428800 bytes (52 MB) copied, 0.202117 s, 259 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102624 s, 511 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.177917 s, 295 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.100112 s, 524 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.323879 s, 162 MB/s
, 0.335873 s, 156 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.329141 s, 159 MB/s
52428800 bytes (52 MB) copied, 0.0943674 s, 556 MB/s
52428800 bytes (52 MB) copied, 0.318893 s, 164 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.186726 s, 281 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.309885 s, 169 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.259906 s, 202 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.332053 s, 158 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.308748 s, 170 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.361225 s, 145 MB/s
52428800 bytes (52 MB) copied, 0.162897 s, 322 MB/s
52428800 bytes (52 MB) copied, 0.160161 s, 327 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.249274 s, 210 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.143372 s, 366 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.423161 s, 124 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.287972 s, 182 MB/s
52428800 bytes (52 MB) copied, 0.178972 s, 293 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.210503 s, 249 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127948 s, 410 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113771 s, 461 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102156 s, 513 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
, 0.0954198 s, 549 MB/s
52428800 bytes (52 MB) copied, 0.0907835 s, 578 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.227361 s, 231 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.266707 s, 197 MB/s
52428800 bytes (52 MB) copied, 0.108857 s, 482 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.156162 s, 336 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.158781 s, 330 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0921224 s, 569 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.201955 s, 260 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.159833 s, 328 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.181427 s, 289 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.18754 s, 280 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0774392 s, 677 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.080027 s, 655 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.197316 s, 266 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.104275 s, 503 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.175597 s, 299 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.127316 s, 412 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.151538 s, 346 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.156527 s, 335 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.214814 s, 244 MB/s
52428800 bytes (52 MB) copied, 0.276868 s, 189 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.327295 s, 160 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.261037 s, 201 MB/s
52428800 bytes (52 MB) copied, 0.328765 s, 159 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.120068 s, 437 MB/s
, 0.319748 s, 164 MB/s
, 0.243442 s, 215 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.393647 s, 133 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.347309 s, 151 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.381424 s, 137 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.380229 s, 138 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.377032 s, 139 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.279391 s, 188 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.241677 s, 217 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.358058 s, 146 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.354615 s, 148 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.256995 s, 204 MB/s
52428800 bytes (52 MB) copied, 0.239417 s, 219 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.550637 s, 95.2 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.266764 s, 197 MB/s
52428800 bytes (52 MB) copied, 0.280239 s, 187 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.222815 s, 235 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.152635 s, 343 MB/s
52428800 bytes (52 MB) copied, 0.415513 s, 126 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.301003 s, 174 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.306007 s, 171 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.201887 s, 260 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.418486 s, 125 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.119302 s, 439 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.162223 s, 323 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153425 s, 342 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149689 s, 350 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.198306 s, 264 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.230713 s, 227 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.225114 s, 233 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.253788 s, 207 MB/s
/mnt/lustre/f45.sanity-pcc
composite_header:
  lcm_magic:         0x0BD60BD0
  lcm_size:          256
  lcm_flags:         ro,pccro
  lcm_layout_gen:    2
  lcm_mirror_count:  2
  lcm_entry_count:   2
components:
  - lcme_id:             65537
    lcme_mirror_id:      1
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         128
    lcme_size:           56
    sub_layout:
      lmm_magic:         0x0BD10BD0
      lmm_seq:           0x2000059f1
      lmm_object_id:     0x8
      lmm_fid:           [0x2000059f1:0x8:0x0]
      lmm_stripe_count:  1
      lmm_stripe_size:   4194304
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 1
      lmm_objects:
      -   0: { l_ost_idx:   1, l_fid: [0x280000400:0x1a5:0x0] }

  - lcme_id:             131074
    lcme_mirror_id:      2
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   EOF
    lcme_offset:         184
    lcme_size:           72
    sub_layout:
      lhm_magic:         0x0BD70BD0
      lhm_pattern:       hsm
      lhm_length:        56
      lhm_type:          0x00000003 (pccro)
      lhm_flags:         exists,archived,pccro,0x109
      lhm_archive_id:    0
      lhm_archive_ver:   0
      lhm_archive_uuid:  ''
llite.lustre-ffff8800a99f2000.stats=
snapshot_time             1746138817.730206591 secs.nsecs
start_time                1746138757.103892452 secs.nsecs
elapsed_time              60.626314139 secs.nsecs
read_bytes                1451 samples [bytes] 278528 1048576 1520435200 1593862900416512
read                      104700 samples [usecs] 166 239701 279738965 9816850671971
ioctl                     1 samples [reqs]
open                      2095 samples [usecs] 5 58739 556679 17961759465
close                     2095 samples [usecs] 5 88919 868110 31573794612
seek                      2093 samples [usecs] 2 33232 53073 1132858383
setattr                   1 samples [usecs] 28571 28571 28571 816302041
getattr                   32 samples [usecs] 121 16843 89768 1101617612
mknod                     1 samples [usecs] 20848 20848 20848 434639104
inode_permission          8370 samples [usecs] 1 52884 1401164 9661123464
opencount                 2096 samples [reqs] 1 2095 2195561 3067197321
openclosetime             2094 samples [usecs] 7 258237 61371427 3703093487333
pcc_attach                1 samples [reqs]
pcc_hit_bytes             2065 samples [bytes] 52428800 52428800 108265472000 5676228778393600000
pcc_attach_bytes          1 samples [bytes] 52428800 52428800 52428800 2748779069440000
llite.lustre-ffff8800b6371000.stats=
snapshot_time             1746138817.730334252 secs.nsecs
start_time                1746138757.103911542 secs.nsecs
elapsed_time              60.626422710 secs.nsecs
read_bytes                1100 samples [bytes] 1048576 1048576 1153433600 1209462790553600
read                      102750 samples [usecs] 169 468541 281511735 11514928958469
open                      2055 samples [usecs] 3 62688 402682 11249890998
close                     2055 samples [usecs] 5 56060 567934 12734754120
seek                      2054 samples [usecs] 2 18027 45549 439431533
inode_permission          4108 samples [usecs] 1 100558 914814 25896704848
opencount                 2055 samples [reqs] 1 2039 2079796 2827807556
openclosetime             2038 samples [usecs] 14 252415 63528659 4391876808851
pcc_attach                1 samples [reqs]
pcc_hit_bytes             2033 samples [bytes] 52428800 52428800 106587750400 5588267848171520000
pcc_attach_bytes          1 samples [bytes] 52428800 52428800 52428800 2748779069440000
attach 2 detach 0 auto_attach 0
llite.lustre-ffff8800a99f2000.pcc_async_threshold=1073741824
llite.lustre-ffff8800b6371000.pcc_async_threshold=1073741824
/mnt/pcc.d45.sanity-pcc.2 is idle now
/mnt/pcc.d45.sanity-pcc is idle now
PASS 45 (69s)

== sanity-pcc test 46: Verify PCC mode setting works correctly ========================================================== 18:33:43 (1746138823)
60+0 records in
60+0 records out
62914560 bytes (63 MB) copied, 0.125696 s, 501 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d46.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/61440           done                            
Creating filesystem with 61440 1k blocks and 15360 inodes
Filesystem UUID: bbfcfc45-cb93-4654-8186-dfdb23aae584
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345

Allocating group tables: 0/8   done                            
Writing inode tables: 0/8   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/8   done

/tmp/f46.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=bbfcfc45-cb93-4654-8186-dfdb23aae584 (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d46.sanity-pcc/d46.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={0}
running as uid/gid/euid/egid 500/500/500/500, groups: 0
 [id]
uid=500(sanityusr) gid=500(sanityusr) groups=500(sanityusr),0(root)
Mode: 0400
llite.lustre-ffff8800a99f2000.pcc_mode=0
running as uid/gid/euid/egid 500/500/500/500, groups: 0
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-r] [/mnt/lustre/f46.sanity-pcc]
lfs pcc pcc: cannot attach the file to PCC with ID 0 failed: Operation not permitted (1)
attach: cannot attach '/mnt/lustre/f46.sanity-pcc' to PCC with attach ID '0': Operation not permitted
running as uid/gid/euid/egid 500/500/500/500, groups: 0
 [cat] [/mnt/lustre/f46.sanity-pcc]
QQQQQ
llite.lustre-ffff8800a99f2000.pcc_mode=0400
running as uid/gid/euid/egid 500/500/500/500, groups: 0
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-r] [/mnt/lustre/f46.sanity-pcc]
lfs pcc pcc: cannot attach the file to PCC with ID 0 failed: Operation not permitted (1)
attach: cannot attach '/mnt/lustre/f46.sanity-pcc' to PCC with attach ID '0': Operation not permitted
running as uid/gid/euid/egid 500/500/500/500, groups: 0
 [cat] [/mnt/lustre/f46.sanity-pcc]
QQQQQ
llite.lustre-ffff8800a99f2000.pcc_mode=0004
running as uid/gid/euid/egid 500/500/500/500, groups: 0
 [cat] [/mnt/lustre/f46.sanity-pcc]
QQQQQ
file: /mnt/lustre/f46.sanity-pcc, type: readonly, PCC_file: /d46.sanity-pcc/59fb/0x2000059f1:0xa:0x0, open_count: 0, flags: 0
running as uid/gid/euid/egid 500/500/500/500, groups: 0
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/f46.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 0
 [stat] [/mnt/lustre/f46.sanity-pcc]
  File: '/mnt/lustre/f46.sanity-pcc'
  Size: 6         	Blocks: 1          IO Block: 4194304 regular file
Device: 2c54f966h/743766374d	Inode: 144115574371254282  Links: 1
Access: (0664/-rw-rw-r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2025-05-01 18:33:45.000000000 -0400
Modify: 2025-05-01 18:33:45.000000000 -0400
Change: 2025-05-01 18:33:45.000000000 -0400
 Birth: -
running as uid/gid/euid/egid 500/500/500/500, groups: 0
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/f46.sanity-pcc]
llite.lustre-ffff8800a99f2000.pcc_mode=0040
running as uid/gid/euid/egid 500/500/500/500, groups: 0
 [cat] [/mnt/lustre/f46.sanity-pcc]
QQQQQ
file: /mnt/lustre/f46.sanity-pcc, type: readonly, PCC_file: /d46.sanity-pcc/59fb/0x2000059f1:0xa:0x0, open_count: 0, flags: 0
running as uid/gid/euid/egid 500/500/500/500, groups: 0
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/f46.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 0
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [attach] [-r] [/mnt/lustre/f46.sanity-pcc]
  File: '/mnt/lustre/f46.sanity-pcc'
  Size: 6         	Blocks: 4          IO Block: 4194304 regular file
Device: 2c54f966h/743766374d	Inode: 144115574371254282  Links: 1
Access: (0660/-rw-rw----)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2025-05-01 18:33:46.000000000 -0400
Modify: 2025-05-01 18:33:45.000000000 -0400
Change: 2025-05-01 18:33:46.000000000 -0400
 Birth: -
file: /mnt/lustre/f46.sanity-pcc, type: readonly, PCC_file: /d46.sanity-pcc/59fb/0x2000059f1:0xa:0x0, open_count: 0, flags: 0
running as uid/gid/euid/egid 500/500/500/500, groups: 0
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [detach] [/mnt/lustre/f46.sanity-pcc]
llite.lustre-ffff8800a99f2000.pcc_mode=0400
llite.lustre-ffff8800a99f2000.pcc_mode=0400
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d46.sanity-pcc is idle now
PASS 46 (6s)

== sanity-pcc test 47: mtime should be kept once file attached into PCC ========================================================== 18:33:50 (1746138830)
60+0 records in
60+0 records out
62914560 bytes (63 MB) copied, 0.125785 s, 500 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d47.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/61440           done                            
Creating filesystem with 61440 1k blocks and 15360 inodes
Filesystem UUID: 0fda1ea3-25ac-4261-bfd0-cefaefc37b42
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345

Allocating group tables: 0/8   done                            
Writing inode tables: 0/8   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/8   done

/tmp/f47.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=0fda1ea3-25ac-4261-bfd0-cefaefc37b42 (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d47.sanity-pcc/d47.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={0}
QQQQQ
/home/green/git/lustre-release/lustre/utils/lfs pcc state /mnt/lustre/f47.sanity-pcc | grep -E -c 'type: readonly'
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d47.sanity-pcc is idle now
PASS 47 (8s)

== sanity-pcc test 48: PCC state should check whether the file in local PCC cache ========================================================== 18:33:58 (1746138838)
60+0 records in
60+0 records out
62914560 bytes (63 MB) copied, 0.140569 s, 448 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d48.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/61440           done                            
Creating filesystem with 61440 1k blocks and 15360 inodes
Filesystem UUID: 677eefbe-fe36-4358-85d5-1c93bb6c3944
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345

Allocating group tables: 0/8   done                            
Writing inode tables: 0/8   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/8   done

/tmp/f48.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=677eefbe-fe36-4358-85d5-1c93bb6c3944 (extents) (64bit) (large files) (huge files)
QQQQQ
file: /mnt/lustre/f48.sanity-pcc, type: readonly, PCC_file: /d48.sanity-pcc/59fd/0x2000059f1:0xc:0x0 (unlinked), open_count: 0, flags: 4
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d48.sanity-pcc is idle now
PASS 48 (6s)

== sanity-pcc test 96: Auto attach from multiple read process on a node ========================================================== 18:34:04 (1746138844)
60+0 records in
60+0 records out
62914560 bytes (63 MB) copied, 0.126236 s, 498 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d96.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/61440           done                            
Creating filesystem with 61440 1k blocks and 15360 inodes
Filesystem UUID: ddc1bff0-5c8b-451d-a181-daeadd80c261
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345

Allocating group tables: 0/8   done                            
Writing inode tables: 0/8   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/8   done

/tmp/f96.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=ddc1bff0-5c8b-451d-a181-daeadd80c261 (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d96.sanity-pcc/d96.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={0}
llite.lustre-ffff8800a99f2000.pcc_async_threshold=1G
llite.lustre-ffff8800b6371000.pcc_async_threshold=1G
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.835839 s, 62.7 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.933896 s, 56.1 MB/s
52428800 bytes (52 MB) copied, 0.944878 s, 55.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.950227 s, 55.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0229593 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0428815 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0568958 s, 921 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0412659 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0449519 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.06362 s, 824 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0437635 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0368739 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0512803 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0457745 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0350937 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0511992 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0459702 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0538947 s, 973 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0626955 s, 836 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0795717 s, 659 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0642671 s, 816 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0538893 s, 973 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0546682 s, 959 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0789173 s, 664 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.109657 s, 478 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0289514 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.065026 s, 806 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0629418 s, 833 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0644814 s, 813 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0475785 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.102687 s, 511 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0654713 s, 801 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0430903 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0552422 s, 949 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0265475 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0288495 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0664043 s, 790 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0463842 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0533439 s, 983 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0486185 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0341712 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0266028 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0415387 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0890527 s, 589 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0305486 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.086796 s, 604 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0550128 s, 953 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258042 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0694776 s, 755 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0501488 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0341757 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0206818 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0315487 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0416731 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0322115 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0764083 s, 686 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0477349 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0420929 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0824764 s, 636 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0345156 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0264654 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0533533 s, 983 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0824564 s, 636 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0449052 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0687194 s, 763 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0505142 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0687997 s, 762 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0620919 s, 844 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0795984 s, 659 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0766089 s, 684 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0770429 s, 681 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0898554 s, 583 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0696313 s, 753 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0607316 s, 863 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0776602 s, 675 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0627262 s, 836 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0595953 s, 880 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0467829 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0737548 s, 711 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0487767 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0401374 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0374422 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0268273 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0389977 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0168906 s, 3.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0694285 s, 755 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0599806 s, 874 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0590601 s, 888 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.059907 s, 875 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0980697 s, 535 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.04284 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0294717 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360462 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0581865 s, 901 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0481898 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0483647 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0596694 s, 879 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0419369 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0422658 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0383827 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0702989 s, 746 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0229096 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.031543 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0324146 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0424285 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0511386 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0470859 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0298819 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0323719 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0489318 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.035996 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0413348 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.03936 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0312865 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375817 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0966257 s, 543 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0895872 s, 585 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0847592 s, 619 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0455477 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0335536 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0341196 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0598673 s, 876 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0733466 s, 715 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0666429 s, 787 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0502964 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.053433 s, 981 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0973864 s, 538 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0960142 s, 546 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0586361 s, 894 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0950053 s, 552 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0651543 s, 805 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.094722 s, 554 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0424186 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0417743 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0712121 s, 736 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0570086 s, 920 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.153477 s, 342 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0972551 s, 539 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.113198 s, 463 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0550484 s, 952 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0496714 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0355417 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0754016 s, 695 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0254252 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103115 s, 508 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0977265 s, 536 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.081673 s, 642 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0288166 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0428726 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0396017 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0244667 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0653381 s, 802 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0570775 s, 919 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0561026 s, 935 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0929105 s, 564 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0660177 s, 794 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0426153 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.108024 s, 485 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0503829 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0625194 s, 839 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0667491 s, 785 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311441 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0581489 s, 902 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0441624 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0428573 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0537176 s, 976 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0356109 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0353617 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0335997 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0816344 s, 642 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0619242 s, 847 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0372067 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0193233 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0216727 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0986256 s, 532 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.116049 s, 452 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0897551 s, 584 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0334025 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0593477 s, 883 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0609026 s, 861 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0383694 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0476907 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0429937 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.051342 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0609067 s, 861 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0560623 s, 935 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0285523 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0719914 s, 728 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0411602 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0222487 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.058177 s, 901 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0539956 s, 971 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0430608 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0332917 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0348716 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.02905 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0329949 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.058404 s, 898 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469155 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0302679 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0731173 s, 717 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0477303 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0518723 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.041459 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0316541 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0458574 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.047527 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0461172 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.068391 s, 767 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311764 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0478655 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0631677 s, 830 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.030902 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0233075 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0756564 s, 693 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0424849 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0473544 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0268463 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0640777 s, 818 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.071414 s, 734 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0855817 s, 613 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0505508 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0572282 s, 916 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0577308 s, 908 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308489 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0359466 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0254499 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0462702 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.026261 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0338816 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0310954 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0435863 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0176475 s, 3.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0241616 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.02935 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0290741 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0652536 s, 803 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0347083 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0484115 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0618239 s, 848 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.030919 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0581926 s, 901 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0423705 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.054586 s, 960 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0495099 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.072926 s, 719 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0459195 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0643328 s, 815 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0316662 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.037969 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375262 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0200737 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306893 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0201331 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0680629 s, 770 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0330286 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0297893 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0323162 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0241594 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0443426 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0711645 s, 737 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.125663 s, 417 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0791379 s, 662 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0592776 s, 884 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0676185 s, 775 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0425018 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0571843 s, 917 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0498193 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.05228 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0351279 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0357782 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0423499 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0500906 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.049513 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0640527 s, 819 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0353784 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.048963 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0710049 s, 738 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0326899 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0242962 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308316 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0205672 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0424526 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0451184 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0301036 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0319658 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0457511 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0269645 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0805433 s, 651 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0610043 s, 859 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0586268 s, 894 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0229153 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0617816 s, 849 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0482304 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.090222 s, 581 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0577474 s, 908 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0760884 s, 689 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0521128 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0443719 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0255655 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0384182 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0448607 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0578226 s, 907 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0434211 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0417875 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0437349 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0686863 s, 763 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0754867 s, 695 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0562252 s, 932 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0316163 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.061138 s, 858 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0510205 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0532839 s, 984 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0577489 s, 908 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.059622 s, 879 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0618964 s, 847 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0334262 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0274812 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0232872 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0323848 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0292506 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0782853 s, 670 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0771109 s, 680 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0319067 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0241524 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0402743 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0412691 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0362721 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.055115 s, 951 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0359308 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0504598 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0313094 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0347208 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0458006 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0250654 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0242183 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0263397 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0379131 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0533181 s, 983 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0531009 s, 987 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0471581 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0501658 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0222657 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0309098 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0226764 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0700339 s, 749 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0794251 s, 660 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0383302 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.041696 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0326118 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0321504 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.043651 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0397809 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0466668 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469755 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0604839 s, 867 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0704381 s, 744 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376806 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0319885 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.040255 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.040315 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0249403 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0383202 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0359738 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0359359 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.043251 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0504583 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0399182 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0575526 s, 911 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0484124 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0242797 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0387319 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0294821 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0299559 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0414186 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0374715 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0440163 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0347491 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0304259 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0291474 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0291398 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0237043 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0310097 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0296696 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0356933 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0509621 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0329056 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0204399 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0270346 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0228567 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0413644 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0268338 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0225965 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0335406 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0261685 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0221581 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0310575 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0249433 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0349614 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0271225 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0255581 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0229744 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0355669 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0322613 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0347674 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0486453 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0198185 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.024209 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.028377 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0365592 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0445115 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0484734 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0309717 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0432862 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0507709 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0368703 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0314145 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0236907 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0424792 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0429475 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0185043 s, 2.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0269202 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0291895 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0260854 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0314294 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0300649 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0314196 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0504315 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0230951 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0276715 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0316581 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0201905 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0477454 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0544955 s, 962 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0510941 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311751 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308332 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0200996 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375796 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376064 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0293608 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0219632 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0245643 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0323328 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306407 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0268542 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0385603 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0390373 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0234988 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0218973 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0682903 s, 768 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0508151 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0441522 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.039598 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0404854 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0643227 s, 815 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469585 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0247127 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370816 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.029214 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0245679 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0252055 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.038089 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0254062 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0313929 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0479053 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0245872 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0282193 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0302454 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.029562 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0221441 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0223284 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0433088 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0260281 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.042007 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0250615 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0290804 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0301768 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0348576 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0316506 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0417249 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.035124 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0536671 s, 977 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.031721 s, 1.7 GB/s
, 0.0391742 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0240002 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.033937 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0456749 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0349909 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352451 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0261031 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0303684 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0359303 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0305994 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0590909 s, 887 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0341144 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0225187 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0257342 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0358728 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0398296 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0304725 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.038622 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110528 s, 474 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0420834 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0511636 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0584805 s, 897 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0464094 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.047355 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0232166 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0567459 s, 924 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258767 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0364955 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0264875 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0591392 s, 887 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0453389 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0517699 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.017576 s, 3.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0667991 s, 785 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0487419 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0416386 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0535185 s, 980 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0471255 s, 1.1 GB/s
52428800 bytes (52 MB) copied, 0.0477996 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0349654 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0543933 s, 964 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0823997 s, 636 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0443661 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0217019 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0318848 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370631 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0467007 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0489201 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0448063 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0356506 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.036025 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0292253 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0273234 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0227913 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0380524 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.035277 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0259507 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0317665 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.03029 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0241649 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0289277 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0260432 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0274548 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0274585 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352208 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0186639 s, 2.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.027212 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0173658 s, 3.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.024979 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0284231 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0239528 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0424518 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0484617 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0625798 s, 838 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0518572 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0322513 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306367 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0342386 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0368794 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0289993 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0440443 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.029298 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0348513 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0189675 s, 2.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0221368 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376658 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306471 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.022027 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0244112 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0230441 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0415963 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0430795 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0390402 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0452266 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0372737 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0303748 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0304184 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0246641 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.043352 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.026296 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0305972 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0345952 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0557693 s, 940 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0238614 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0586802 s, 893 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0362885 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0460775 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0282453 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0412867 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0264164 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0266029 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0436199 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0212421 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0461263 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0440905 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0447713 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0739384 s, 709 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0279573 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0337393 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0276129 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0374911 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0581734 s, 901 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0431164 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0569948 s, 920 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0521921 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0346427 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0580926 s, 903 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0404677 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0302798 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0401923 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0703786 s, 745 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287977 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0303767 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0306581 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0299427 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0579827 s, 904 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0668493 s, 784 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0496238 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0275431 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0262551 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0208277 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0368033 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0517589 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0349818 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0340072 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0264366 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0261884 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0233437 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0305464 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0403983 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258753 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0315116 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0385108 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0618289 s, 848 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0276176 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0258732 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0314562 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0299803 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0254938 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0361477 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0233809 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0355852 s, 1.5 GB/s
, 0.0296612 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0224666 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0334235 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352831 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0319976 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0359309 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311297 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0201322 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0274831 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.058749 s, 892 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0439234 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0214618 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0507806 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0462029 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0438717 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336051 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0308632 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.028081 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0220216 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0296243 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0245545 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0473123 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0255537 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0209374 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311481 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0249444 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.025321 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0374657 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0278215 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0530487 s, 988 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0224221 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0324496 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0367974 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0399503 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0340717 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0397334 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0318 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0252633 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0368371 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0299767 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0435009 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0321846 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0585199 s, 896 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0624476 s, 840 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0282445 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.054972 s, 954 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0367178 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0420491 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0503946 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0555703 s, 943 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0353382 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0333614 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0263338 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0320798 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0248691 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0632637 s, 829 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0291913 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0441474 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360578 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0254626 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0344486 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0322252 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0392063 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0257613 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0313898 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0558106 s, 939 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0436765 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0301858 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0360694 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0343645 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0251101 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0247283 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0490989 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0409005 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0263586 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0293166 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0231538 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.045442 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0471637 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0290684 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0386303 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0570011 s, 920 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0709543 s, 739 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0332849 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0359789 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.033802 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375763 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0354346 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0554732 s, 945 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0327831 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0210304 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0380339 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0260924 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0210387 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0288365 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0488826 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.028805 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0531545 s, 986 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0597023 s, 878 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.037751 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0259905 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0607176 s, 863 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0317364 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376186 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0256824 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0309928 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0348758 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0290835 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0581932 s, 901 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522447 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.06703 s, 782 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0384059 s, 1.4 GB/s
52428800 bytes (52 MB) copied, 0.039456 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0324943 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0443964 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0439497 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0301629 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.02949 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0284018 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0267436 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0356081 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0362658 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0270617 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0263363 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0439377 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0461218 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0557859 s, 940 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0405421 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.065336 s, 802 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0537791 s, 975 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0624018 s, 840 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0298002 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0499265 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0348446 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0486011 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0395143 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0497717 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0215252 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0339023 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0384079 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0320298 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0357708 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0285281 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.043274 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0238502 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0362949 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0486285 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0315028 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0518952 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0236899 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0425857 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0424814 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0267259 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0462972 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0588267 s, 891 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0432883 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0367602 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0336358 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0534917 s, 980 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0442554 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.03344 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0314392 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0520362 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0603667 s, 869 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0251428 s, 2.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0700742 s, 748 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0220241 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.018011 s, 2.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0345662 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0281777 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0785612 s, 667 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0408299 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0547285 s, 958 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0846135 s, 620 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0241945 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0282376 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0309598 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0199795 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0611082 s, 858 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0300489 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0319463 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375791 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0340375 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0262962 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0640652 s, 818 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0329149 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0341465 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0260602 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0307633 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0373931 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0316613 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0423595 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352934 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0223345 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0401512 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0218499 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0587653 s, 892 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0396752 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0576535 s, 909 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0391558 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0664134 s, 789 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0207177 s, 2.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0428173 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0475548 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0406275 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0282073 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0402923 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0403204 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0536654 s, 977 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0477853 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0337059 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0501568 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0396202 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0717972 s, 730 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0398365 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0365563 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0697037 s, 752 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0720864 s, 727 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0371618 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0522757 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0390785 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0598639 s, 876 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0377566 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.04428 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0285441 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0945923 s, 554 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0452273 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.059955 s, 874 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0703597 s, 745 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0538877 s, 973 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.031338 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0829477 s, 632 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.154559 s, 339 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0721993 s, 726 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0560024 s, 936 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0721781 s, 726 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0289389 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0475843 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0335903 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0331211 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.036432 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0497946 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0467221 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0557989 s, 940 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0402759 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0605141 s, 866 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0510647 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0311574 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0266335 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0310096 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.031309 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0493725 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0234098 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0686442 s, 764 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0366255 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0789651 s, 664 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.117581 s, 446 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.152216 s, 344 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0476093 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.029619 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.026778 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.032834 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0281029 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0451514 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0288051 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0726113 s, 722 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0194143 s, 2.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0431629 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0560613 s, 935 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0603596 s, 869 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0548415 s, 956 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0513208 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0292486 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0626244 s, 837 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0514646 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0282201 s, 1.9 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0731282 s, 717 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.105118 s, 499 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0750308 s, 699 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0722039 s, 726 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0339226 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0338276 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0290892 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0348298 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0445158 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0459133 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0302019 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0486482 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.037211 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0470418 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0316738 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0318796 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0545293 s, 961 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0666864 s, 786 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0945001 s, 555 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.081093 s, 647 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0688865 s, 761 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0571344 s, 918 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.041878 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0588658 s, 891 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0577139 s, 908 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0322622 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0407687 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0328423 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0253055 s, 2.1 GB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.0387967 s, 1.4 GB/s
, 0.0306764 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.044693 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0314098 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0471276 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0521445 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0393619 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0761627 s, 688 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0243806 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0478172 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0357039 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0479609 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.020454 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0350168 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352415 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0390322 s, 1.3 GB/s
Finish ========
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0320559 s, 1.6 GB/s
detach: must specify one or more FIDs
Detach given files from the Persistent Client Cache.
usage: lfs pcc detach [-k] FILE [...]

lfs commands used to interact with PCC features:
lfs pcc attach - attach given files to Persistent Client Cache
lfs pcc attach_fid - attach given files into PCC by FID(s)
lfs pcc state  - display the PCC state for given files
lfs pcc detach - detach given files from Persistent Client Cache
lfs pcc detach_fid - detach given files from PCC by FID(s)
lfs pcc delete - delete the PCC layout componenet for given files
lfs pcc pin - pin give files for PCC
lfs pcc unpin - unpin given files for PCC

llite.lustre-ffff8800a99f2000.pcc_async_threshold=1073741824
llite.lustre-ffff8800b6371000.pcc_async_threshold=1073741824
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d96.sanity-pcc is idle now
PASS 96 (68s)

== sanity-pcc test 97: two mmap I/O and layout lock cancel ========================================================== 18:35:12 (1746138912)
60+0 records in
60+0 records out
62914560 bytes (63 MB) copied, 0.109037 s, 577 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d97.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/61440           done                            
Creating filesystem with 61440 1k blocks and 15360 inodes
Filesystem UUID: 710a43c8-6080-446a-a98c-b8f378901eee
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345

Allocating group tables: 0/8   done                            
Writing inode tables: 0/8   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/8   done

/tmp/f97.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=710a43c8-6080-446a-a98c-b8f378901eee (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d97.sanity-pcc/d97.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={0}
llite.lustre-ffff8800a99f2000.pcc_async_threshold=1G
llite.lustre-ffff8800b6371000.pcc_async_threshold=1G
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.546672 s, 95.9 MB/s
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T1. mmap_cat /mnt/lustre/f97.sanity-pcc ...
T2. mmap_cat /mnt/lustre/f97.sanity-pcc ...
llite.lustre-ffff8800a99f2000.pcc_async_threshold=1073741824
llite.lustre-ffff8800b6371000.pcc_async_threshold=1073741824
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d97.sanity-pcc is idle now
PASS 97 (127s)

== sanity-pcc test 98: racer between auto attach and mmap I/O ========================================================== 18:37:19 (1746139039)
60+0 records in
60+0 records out
62914560 bytes (63 MB) copied, 0.110158 s, 571 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d98.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/61440           done                            
Creating filesystem with 61440 1k blocks and 15360 inodes
Filesystem UUID: 50e6b517-669f-42d8-b1b1-2defc1b99b00
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345

Allocating group tables: 0/8   done                            
Writing inode tables: 0/8   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/8   done

/tmp/f98.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=50e6b517-669f-42d8-b1b1-2defc1b99b00 (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d98.sanity-pcc/d98.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={0}
llite.lustre-ffff8800a99f2000.pcc_async_threshold=0
llite.lustre-ffff8800b6371000.pcc_async_threshold=0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.643142 s, 81.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.774863 s, 67.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.09701 s, 47.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.32067 s, 39.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0241197 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.111369 s, 471 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.743712 s, 70.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.743964 s, 70.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0468269 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.055652 s, 942 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0351136 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0819423 s, 640 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0738408 s, 710 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0237194 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0340266 s, 1.5 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0603728 s, 868 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0203653 s, 2.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0512809 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0647045 s, 810 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0741923 s, 707 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0947358 s, 553 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0314698 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0458018 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.17241 s, 44.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.32729 s, 39.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.03342 s, 50.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.428297 s, 122 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.656034 s, 79.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.494547 s, 106 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.06036 s, 49.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.20635 s, 43.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.20595 s, 43.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.76078 s, 68.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.00891 s, 52.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.6837 s, 76.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.38202 s, 37.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.33386 s, 39.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.436664 s, 120 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.50443 s, 104 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 2.02749 s, 25.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.02246 s, 51.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.828973 s, 63.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.03386 s, 50.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.496507 s, 106 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.565157 s, 92.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.351469 s, 149 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.717931 s, 73.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.548282 s, 95.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.73045 s, 71.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.290602 s, 180 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.193381 s, 271 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0743487 s, 705 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.83853 s, 62.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.713887 s, 73.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.71654 s, 30.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.618231 s, 84.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.868979 s, 60.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0830807 s, 631 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.648524 s, 80.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.581129 s, 90.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.644526 s, 81.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.464013 s, 113 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.478023 s, 110 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.491767 s, 107 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.786356 s, 66.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.788238 s, 66.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.58437 s, 89.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.567145 s, 92.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.614136 s, 85.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.740615 s, 70.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.08778 s, 48.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.56456 s, 33.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.504114 s, 104 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 2.1027 s, 24.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0692183 s, 757 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0863895 s, 607 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0968519 s, 541 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0686262 s, 764 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0264563 s, 2.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0768942 s, 682 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0373986 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0670136 s, 782 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0790609 s, 663 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0837451 s, 626 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0621432 s, 844 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0484684 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0924328 s, 567 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0913207 s, 574 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0793341 s, 661 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0495459 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0542889 s, 966 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0312567 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0434838 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0226281 s, 2.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0829957 s, 632 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0632333 s, 829 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0899022 s, 583 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0395881 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.077065 s, 680 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0569618 s, 920 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0624763 s, 839 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0523053 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.112189 s, 467 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0610725 s, 858 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0910725 s, 576 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0866146 s, 605 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0903644 s, 580 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0758982 s, 691 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0405028 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0750486 s, 699 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0575047 s, 912 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0716602 s, 732 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0651392 s, 805 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0394673 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0701624 s, 747 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0505336 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0391416 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.809328 s, 64.8 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.954662 s, 54.9 MB/s
52428800 bytes (52 MB) copied, 0.951376 s, 55.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.872526 s, 60.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.0177 s, 51.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.721288 s, 72.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.536048 s, 97.8 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.687014 s, 76.3 MB/s
52428800 bytes (52 MB) copied, 0.646721 s, 81.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.71151 s, 30.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.77578 s, 29.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.83872 s, 28.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.324824 s, 161 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0985855 s, 532 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.324023 s, 162 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0287376 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.101644 s, 516 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0705395 s, 743 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0304006 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0479363 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0430293 s, 1.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0714489 s, 734 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0654875 s, 801 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0595388 s, 881 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0618894 s, 847 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375796 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0738587 s, 710 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0849786 s, 617 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0476815 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0507113 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0375603 s, 1.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0626451 s, 837 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0416384 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.079271 s, 661 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0669927 s, 783 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0997225 s, 526 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0578192 s, 907 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0298148 s, 1.8 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0693467 s, 756 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0307418 s, 1.7 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0399333 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0667086 s, 786 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0234099 s, 2.2 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0391411 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.904489 s, 58.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.909222 s, 57.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.912325 s, 57.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.69693 s, 75.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.567907 s, 92.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.704733 s, 74.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.247963 s, 211 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0571668 s, 917 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.213856 s, 245 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.2093 s, 43.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.00643 s, 52.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.20776 s, 43.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.769662 s, 68.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.872576 s, 60.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.958655 s, 54.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.454854 s, 115 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.498426 s, 105 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.530416 s, 98.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.424737 s, 123 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.423049 s, 124 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.275688 s, 190 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.856808 s, 61.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.975958 s, 53.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.614477 s, 85.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.95331 s, 55.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.816636 s, 64.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.06425 s, 49.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.871499 s, 60.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.06878 s, 49.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.788854 s, 66.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.659252 s, 79.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.62921 s, 83.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.717082 s, 73.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.199673 s, 263 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.474793 s, 110 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.480179 s, 109 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.634972 s, 82.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.732232 s, 71.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.318751 s, 164 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.142883 s, 367 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.146116 s, 359 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.367575 s, 143 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.632304 s, 82.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.713564 s, 73.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.353738 s, 148 MB/s
50+0 records in
50+0 records out
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied52428800 bytes (52 MB) copied, 0.718583 s, 73.0 MB/s
, 0.720554 s, 72.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.912972 s, 57.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.671098 s, 78.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.602974 s, 87.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.1923 s, 44.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.282083 s, 186 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.167493 s, 313 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0325859 s, 1.6 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0519872 s, 1.0 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0415028 s, 1.3 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0593782 s, 883 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103832 s, 505 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0792788 s, 661 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0469838 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0684491 s, 766 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0259814 s, 2.0 GB/s
llite.lustre-ffff8800a99f2000.pcc_async_threshold=1073741824
llite.lustre-ffff8800b6371000.pcc_async_threshold=1073741824
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d98.sanity-pcc is idle now
PASS 98 (70s)

== sanity-pcc test 99: race among unlink | mmap read | write | detach for PCC-RO file ========================================================== 18:38:29 (1746139109)
60+0 records in
60+0 records out
62914560 bytes (63 MB) copied, 0.152016 s, 414 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d99.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/61440           done                            
Creating filesystem with 61440 1k blocks and 15360 inodes
Filesystem UUID: 5bd2f117-d6d7-4717-b9d8-c35c8d0b009a
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345

Allocating group tables: 0/8   done                            
Writing inode tables: 0/8   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/8   done

/tmp/f99.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=5bd2f117-d6d7-4717-b9d8-c35c8d0b009a (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={0}
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.942067 s, 55.7 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 2.02005 s, 26.0 MB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0242639 s, 2.2 GB/s
Detach /mnt/lustre/f99.sanity-pcc ...
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.23846 s, 42.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.35329 s, 38.7 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.1283 s, 46.5 MB/s
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0691652 s, 758 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.771641 s, 67.9 MB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.959913 s, 54.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.11709 s, 46.9 MB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.49492 s, 35.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0616853 s, 850 MB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Detach /mnt/lustre/f99.sanity-pcc ...
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.938219 s, 55.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.23553 s, 42.4 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.995448 s, 52.7 MB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.32039 s, 39.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.51042 s, 34.7 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.05904 s, 49.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370342 s, 1.4 GB/s
Detach /mnt/lustre/f99.sanity-pcc ...
Read /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.0788 s, 48.6 MB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.149118 s, 352 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.408276 s, 128 MB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.00128 s, 52.4 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.04937 s, 50.0 MB/s
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0328429 s, 1.6 GB/s
Detach /mnt/lustre/f99.sanity-pcc ...
Read /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.918597 s, 57.1 MB/s
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0242057 s, 2.2 GB/s
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.591829 s, 88.6 MB/s
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.34294 s, 39.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0190761 s, 2.7 GB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Read /mnt/lustre/f99.sanity-pcc ...
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.852417 s, 61.5 MB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.924557 s, 56.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.0904 s, 48.1 MB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Read /mnt/lustre/f99.sanity-pcc ...
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.0716 s, 48.9 MB/s
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0548495 s, 956 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.955944 s, 54.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.28623 s, 40.8 MB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.976674 s, 53.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0324092 s, 1.6 GB/s
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.21433 s, 43.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0194361 s, 2.7 GB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Detach /mnt/lustre/f99.sanity-pcc ...
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.04664 s, 50.1 MB/s
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.849327 s, 61.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0352962 s, 1.5 GB/s
Detach /mnt/lustre/f99.sanity-pcc ...
Read /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.883224 s, 59.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.848212 s, 61.8 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0886058 s, 592 MB/s
Detach /mnt/lustre/f99.sanity-pcc ...
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.838118 s, 62.6 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.34539 s, 39.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0415328 s, 1.3 GB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.883441 s, 59.3 MB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.02783 s, 51.0 MB/s
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.069 s, 49.0 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0558203 s, 939 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.434397 s, 121 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.27155 s, 41.2 MB/s
Detach /mnt/lustre/f99.sanity-pcc ...
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.41596 s, 37.0 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.71462 s, 30.6 MB/s
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.08654 s, 48.3 MB/s
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0597277 s, 878 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.916262 s, 57.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.06398 s, 49.3 MB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Read /mnt/lustre/f99.sanity-pcc ...
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.36428 s, 38.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0759036 s, 691 MB/s
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Read /mnt/lustre/f99.sanity-pcc ...
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.63664 s, 32.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.8929 s, 27.7 MB/s
Read /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Detach /mnt/lustre/f99.sanity-pcc ...
Unlink /mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0
unlink: cannot unlink '/mnt/pcc.d99.sanity-pcc/d99.sanity-pcc/59e2/0x2000059f1:0x13:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.65644 s, 31.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0525054 s, 999 MB/s
Detach /mnt/lustre/f99.sanity-pcc ...
Read /mnt/lustre/f99.sanity-pcc ...
==== DONE ====
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.08628 s, 48.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.20731 s, 253 MB/s
==== DONE WIAT ====
osc.lustre-OST0000-osc-ffff8800a99f2000.rpc_stats=
snapshot_time:            1746139174.316717698 secs.nsecs
start_time:               1746138600.509842429 secs.nsecs
elapsed_time:             573.806875269 secs.nsecs
read RPCs in flight:  0
write RPCs in flight: 0
DIO RPCs in flight: 0
pending write pages:  0
pending read pages:   0

			read			write
pages per rpc         rpcs   % cum % |       rpcs   % cum %
1:		       927   7   7   |          3   0   0
2:		         7   0   7   |          0   0   0
4:		         0   0   7   |          0   0   0
8:		        13   0   8   |          0   0   0
16:		      2526  21  29   |          0   0   0
32:		       294   2  32   |          0   0   0
64:		       458   3  35   |          0   0   0
128:		       978   8  44   |          0   0   0
256:		      6543  55 100   |       2050  99 100

			read			write
rpcs in flight        rpcs   % cum % |       rpcs   % cum %
1:		      7818  66  66   |        240  11  11
2:		       967   8  74   |        513  24  36
3:		       392   3  78   |        453  22  58
4:		       319   2  80   |        305  14  73
5:		       278   2  83   |        191   9  82
6:		       217   1  85   |        149   7  90
7:		       220   1  86   |         74   3  93
8:		       215   1  88   |         58   2  96
9:		      1153   9  98   |         68   3  99
10:		        18   0  98   |          2   0 100
11:		        12   0  98   |          0   0 100
12:		        10   0  98   |          0   0 100
13:		        10   0  99   |          0   0 100
14:		        10   0  99   |          0   0 100
15:		        10   0  99   |          0   0 100
16:		        10   0  99   |          0   0 100
17:		         9   0  99   |          0   0 100
18:		         9   0  99   |          0   0 100
19:		         9   0  99   |          0   0 100
20:		         7   0  99   |          0   0 100
21:		         7   0  99   |          0   0 100
22:		         8   0  99   |          0   0 100
23:		         6   0  99   |          0   0 100
24:		         6   0  99   |          0   0 100
25:		         5   0  99   |          0   0 100
26:		         5   0  99   |          0   0 100
27:		         5   0  99   |          0   0 100
28:		         4   0  99   |          0   0 100
29:		         4   0  99   |          0   0 100
30:		         1   0  99   |          0   0 100
31:		         2   0 100   |          0   0 100

			read			write
offset                rpcs   % cum % |       rpcs   % cum %
0:		       129   1   1   |         44   2   2
1:		        52   0   1   |          0   0   2
2:		       102   0   2   |          0   0   2
4:		         9   0   2   |          0   0   2
8:		         0   0   2   |          0   0   2
16:		       101   0   3   |          0   0   2
32:		       161   1   4   |          0   0   2
64:		       282   2   7   |          0   0   2
128:		       531   4  11   |          0   0   2
256:		       261   2  13   |         41   1   4
512:		       546   4  18   |         82   3   8
1024:		       941   8  26   |        164   7  16
2048:		      1811  15  41   |        328  15  32
4096:		      3216  27  69   |        656  31  64
8192:		      3604  30 100   |        738  35 100
osc.lustre-OST0000-osc-ffff8800b6371000.rpc_stats=
snapshot_time:            1746139174.316858963 secs.nsecs
start_time:               1746138602.514516043 secs.nsecs
elapsed_time:             571.802342920 secs.nsecs
read RPCs in flight:  0
write RPCs in flight: 0
DIO RPCs in flight: 0
pending write pages:  0
pending read pages:   0

			read			write
pages per rpc         rpcs   % cum % |       rpcs   % cum %
1:		         0   0   0   |          0   0   0
2:		         0   0   0   |          0   0   0
4:		         0   0   0   |          0   0   0
8:		         0   0   0   |          0   0   0
16:		         0   0   0   |          0   0   0
32:		         0   0   0   |          0   0   0
64:		         0   0   0   |          0   0   0
128:		         0   0   0   |          0   0   0
256:		        50 100 100   |          0   0   0

			read			write
rpcs in flight        rpcs   % cum % |       rpcs   % cum %
1:		         3   6   6   |          0   0   0
2:		         1   2   8   |          0   0   0
3:		         1   2  10   |          0   0   0
4:		         1   2  12   |          0   0   0
5:		         1   2  14   |          0   0   0
6:		         1   2  16   |          0   0   0
7:		         1   2  18   |          0   0   0
8:		         1   2  20   |          0   0   0
9:		        40  80 100   |          0   0   0

			read			write
offset                rpcs   % cum % |       rpcs   % cum %
0:		         1   2   2   |          0   0   0
1:		         0   0   2   |          0   0   0
2:		         0   0   2   |          0   0   0
4:		         0   0   2   |          0   0   0
8:		         0   0   2   |          0   0   0
16:		         0   0   2   |          0   0   0
32:		         0   0   2   |          0   0   0
64:		         0   0   2   |          0   0   0
128:		         0   0   2   |          0   0   0
256:		         1   2   4   |          0   0   0
512:		         2   4   8   |          0   0   0
1024:		         4   8  16   |          0   0   0
2048:		         8  16  32   |          0   0   0
4096:		        16  32  64   |          0   0   0
8192:		        18  36 100   |          0   0   0
osc.lustre-OST0001-osc-ffff8800a99f2000.rpc_stats=
snapshot_time:            1746139174.316945284 secs.nsecs
start_time:               1746138600.510640942 secs.nsecs
elapsed_time:             573.806304342 secs.nsecs
read RPCs in flight:  0
write RPCs in flight: 0
DIO RPCs in flight: 0
pending write pages:  0
pending read pages:   0

			read			write
pages per rpc         rpcs   % cum % |       rpcs   % cum %
1:		         3   0   0   |          3   1   1
2:		         0   0   0   |          0   0   1
4:		         0   0   0   |          0   0   1
8:		         0   0   0   |          0   0   1
16:		         0   0   0   |          0   0   1
32:		         0   0   0   |          0   0   1
64:		         0   0   0   |          0   0   1
128:		         0   0   0   |          0   0   1
256:		       352  99 100   |        150  98 100

			read			write
rpcs in flight        rpcs   % cum % |       rpcs   % cum %
1:		        37  10  10   |         17  11  11
2:		        27   7  18   |         89  58  69
3:		        19   5  23   |         38  24  94
4:		        18   5  28   |          8   5  99
5:		        13   3  32   |          1   0 100
6:		        13   3  35   |          0   0 100
7:		        13   3  39   |          0   0 100
8:		        20   5  45   |          0   0 100
9:		       195  54 100   |          0   0 100

			read			write
offset                rpcs   % cum % |       rpcs   % cum %
0:		        12   3   3   |          6   3   3
1:		         0   0   3   |          0   0   3
2:		         0   0   3   |          0   0   3
4:		         0   0   3   |          0   0   3
8:		         0   0   3   |          0   0   3
16:		         0   0   3   |          0   0   3
32:		         0   0   3   |          0   0   3
64:		         0   0   3   |          0   0   3
128:		         0   0   3   |          0   0   3
256:		         7   1   5   |          3   1   5
512:		        14   3   9   |          6   3   9
1024:		        28   7  17   |         12   7  17
2048:		        56  15  32   |         24  15  33
4096:		       112  31  64   |         48  31  64
8192:		       126  35 100   |         54  35 100
osc.lustre-OST0001-osc-ffff8800b6371000.rpc_stats=
snapshot_time:            1746139174.317017956 secs.nsecs
start_time:               1746138602.514919441 secs.nsecs
elapsed_time:             571.802098515 secs.nsecs
read RPCs in flight:  0
write RPCs in flight: 0
DIO RPCs in flight: 0
pending write pages:  0
pending read pages:   0

			read			write
pages per rpc         rpcs   % cum % |       rpcs   % cum %
1:		         0   0   0   |          0   0   0
2:		         0   0   0   |          0   0   0
4:		         0   0   0   |          0   0   0
8:		         0   0   0   |          0   0   0
16:		         0   0   0   |          0   0   0
32:		         0   0   0   |          0   0   0
64:		         0   0   0   |          0   0   0
128:		         0   0   0   |          0   0   0
256:		        50 100 100   |          0   0   0

			read			write
rpcs in flight        rpcs   % cum % |       rpcs   % cum %
1:		         2   4   4   |          0   0   0
2:		         1   2   6   |          0   0   0
3:		         1   2   8   |          0   0   0
4:		         1   2  10   |          0   0   0
5:		         1   2  12   |          0   0   0
6:		         1   2  14   |          0   0   0
7:		         2   4  18   |          0   0   0
8:		         2   4  22   |          0   0   0
9:		        39  78 100   |          0   0   0

			read			write
offset                rpcs   % cum % |       rpcs   % cum %
0:		         1   2   2   |          0   0   0
1:		         0   0   2   |          0   0   0
2:		         0   0   2   |          0   0   0
4:		         0   0   2   |          0   0   0
8:		         0   0   2   |          0   0   0
16:		         0   0   2   |          0   0   0
32:		         0   0   2   |          0   0   0
64:		         0   0   2   |          0   0   0
128:		         0   0   2   |          0   0   0
256:		         1   2   4   |          0   0   0
512:		         2   4   8   |          0   0   0
1024:		         4   8  16   |          0   0   0
2048:		         8  16  32   |          0   0   0
4096:		        16  32  64   |          0   0   0
8192:		        18  36 100   |          0   0   0
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d99.sanity-pcc is idle now
PASS 99 (71s)

== sanity-pcc test 99b: race among unlink | two readers | write | detach for PCC-RO file ========================================================== 18:39:40 (1746139180)
200+0 records in
200+0 records out
209715200 bytes (210 MB) copied, 0.554805 s, 378 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d99b.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:   1024/204800             done                            
Creating filesystem with 204800 1k blocks and 51200 inodes
Filesystem UUID: 3806976a-9577-4967-9402-77e8955368f3
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345, 73729

Allocating group tables:  0/25     done                            
Writing inode tables:  0/25     done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information:  0/25     done

/tmp/f99b.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=3806976a-9577-4967-9402-77e8955368f3 (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={0}
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.786282 s, 66.7 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
Detach /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.35061 s, 38.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.47189 s, 35.6 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0832235 s, 630 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copiedDetach /mnt/lustre/f99b.sanity-pcc ...
, 0.793654 s, 66.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.21625 s, 43.1 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
Detach /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.43792 s, 36.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.53343 s, 34.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0291616 s, 1.8 GB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.55128 s, 33.8 MB/s
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.78464 s, 29.4 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0310753 s, 1.7 GB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0376629 s, 1.4 GB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.46499 s, 35.8 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
Read /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0979676 s, 535 MB/s
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.79185 s, 29.3 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
Read /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0275732 s, 1.9 GB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.936893 s, 56.0 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.17415 s, 44.7 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.619056 s, 84.7 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.48001 s, 35.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.68142 s, 31.2 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.84103 s, 62.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0577402 s, 908 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Read /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.39456 s, 37.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.678945 s, 77.2 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.036241 s, 1.4 GB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.67669 s, 31.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.103626 s, 506 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.318726 s, 164 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.032048 s, 1.6 GB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Read /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.2071 s, 43.4 MB/s
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.653599 s, 80.2 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.675414 s, 77.6 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.25532 s, 41.8 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.04727 s, 50.1 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.0111 s, 51.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.031352 s, 1.7 GB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
Read /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.782968 s, 67.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.786327 s, 66.7 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.20769 s, 43.4 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.575677 s, 91.1 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.03977 s, 50.4 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.758724 s, 69.1 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.870462 s, 60.2 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.9315 s, 56.3 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.17101 s, 44.8 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Read /mnt/lustre/f99b.sanity-pcc ...
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.688144 s, 76.2 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.919873 s, 57.0 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
Read /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0760161 s, 690 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.49486 s, 35.1 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.10417 s, 47.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.09413 s, 47.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0289259 s, 1.8 GB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
Read /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.894337 s, 58.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.14584 s, 45.8 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.13729 s, 46.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.881916 s, 59.4 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.849805 s, 61.7 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.982156 s, 53.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.25134 s, 41.9 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.22489 s, 42.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.890499 s, 58.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.049359 s, 1.1 GB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.884712 s, 59.3 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.838266 s, 62.5 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
Read /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.308894 s, 170 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.39721 s, 132 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.19918 s, 43.7 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0752009 s, 697 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.664042 s, 79.0 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
Read /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.0222 s, 51.3 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.34092 s, 39.1 MB/s
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.1461 s, 45.7 MB/s
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.01234 s, 51.8 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0664145 s, 789 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.40782 s, 37.2 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.814543 s, 64.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0786725 s, 666 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.806963 s, 65.0 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.651862 s, 80.4 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.512342 s, 102 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.880798 s, 59.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0510513 s, 1.0 GB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.11971 s, 46.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.25209 s, 41.9 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.07556 s, 48.7 MB/s
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
unlink: cannot unlink '/mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0': No such file or directory
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.10291 s, 47.5 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0845026 s, 620 MB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.583281 s, 89.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.824061 s, 63.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.028663 s, 1.8 GB/s
Read /mnt/lustre/f99b.sanity-pcc ...
Detach /mnt/lustre/f99b.sanity-pcc ...
Unlink /mnt/pcc.d99b.sanity-pcc/d99b.sanity-pcc/59db/0x2000059f1:0x2a:0x0
Detach /mnt/lustre/f99b.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.45776 s, 36.0 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.31919 s, 39.7 MB/s
==== DONE ====
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.048657 s, 1.1 GB/s
==== DONE WIAT ====
osc.lustre-OST0000-osc-ffff8800a99f2000.rpc_stats=
snapshot_time:            1746139244.987895430 secs.nsecs
start_time:               1746138600.509842429 secs.nsecs
elapsed_time:             644.478053001 secs.nsecs
read RPCs in flight:  0
write RPCs in flight: 0
DIO RPCs in flight: 0
pending write pages:  0
pending read pages:   0

			read			write
pages per rpc         rpcs   % cum % |       rpcs   % cum %
1:		       927   6   6   |          3   0   0
2:		         7   0   6   |          0   0   0
4:		         0   0   6   |          0   0   0
8:		        13   0   6   |          0   0   0
16:		      2526  18  25   |          0   0   0
32:		       294   2  27   |          0   0   0
64:		       458   3  30   |          0   0   0
128:		       978   7  37   |          0   0   0
256:		      8641  62 100   |       3700  99 100

			read			write
rpcs in flight        rpcs   % cum % |       rpcs   % cum %
1:		      8149  58  58   |        397  10  10
2:		      1206   8  67   |        879  23  34
3:		       573   4  71   |        797  21  55
4:		       521   3  75   |        554  14  70
5:		       473   3  78   |        382  10  81
6:		       406   2  81   |        284   7  88
7:		       403   2  84   |        152   4  93
8:		       374   2  87   |        109   2  95
9:		      1569  11  98   |        146   3  99
10:		        20   0  98   |          3   0 100
11:		        13   0  99   |          0   0 100
12:		        10   0  99   |          0   0 100
13:		        10   0  99   |          0   0 100
14:		        10   0  99   |          0   0 100
15:		        10   0  99   |          0   0 100
16:		        10   0  99   |          0   0 100
17:		         9   0  99   |          0   0 100
18:		         9   0  99   |          0   0 100
19:		         9   0  99   |          0   0 100
20:		         7   0  99   |          0   0 100
21:		         7   0  99   |          0   0 100
22:		         8   0  99   |          0   0 100
23:		         6   0  99   |          0   0 100
24:		         6   0  99   |          0   0 100
25:		         5   0  99   |          0   0 100
26:		         5   0  99   |          0   0 100
27:		         5   0  99   |          0   0 100
28:		         4   0  99   |          0   0 100
29:		         4   0  99   |          0   0 100
30:		         1   0  99   |          0   0 100
31:		         2   0 100   |          0   0 100

			read			write
offset                rpcs   % cum % |       rpcs   % cum %
0:		       178   1   1   |         77   2   2
1:		        52   0   1   |          0   0   2
2:		       102   0   2   |          0   0   2
4:		         9   0   2   |          0   0   2
8:		         0   0   2   |          0   0   2
16:		       101   0   3   |          0   0   2
32:		       161   1   4   |          0   0   2
64:		       282   2   6   |          0   0   2
128:		       531   3  10   |          0   0   2
256:		       307   2  12   |         74   1   4
512:		       632   4  17   |        148   3   8
1024:		      1113   8  25   |        296   7  16
2048:		      2142  15  40   |        592  15  32
4096:		      3864  27  68   |       1184  31  64
8192:		      4370  31 100   |       1332  35 100
osc.lustre-OST0000-osc-ffff8800b6371000.rpc_stats=
snapshot_time:            1746139244.988291215 secs.nsecs
start_time:               1746138602.514516043 secs.nsecs
elapsed_time:             642.473775172 secs.nsecs
read RPCs in flight:  0
write RPCs in flight: 0
DIO RPCs in flight: 0
pending write pages:  0
pending read pages:   0

			read			write
pages per rpc         rpcs   % cum % |       rpcs   % cum %
1:		         0   0   0   |          0   0   0
2:		         0   0   0   |          0   0   0
4:		         0   0   0   |          0   0   0
8:		         0   0   0   |          0   0   0
16:		         0   0   0   |          0   0   0
32:		         0   0   0   |          0   0   0
64:		         0   0   0   |          0   0   0
128:		         0   0   0   |          0   0   0
256:		        50 100 100   |          0   0   0

			read			write
rpcs in flight        rpcs   % cum % |       rpcs   % cum %
1:		         3   6   6   |          0   0   0
2:		         1   2   8   |          0   0   0
3:		         1   2  10   |          0   0   0
4:		         1   2  12   |          0   0   0
5:		         1   2  14   |          0   0   0
6:		         1   2  16   |          0   0   0
7:		         1   2  18   |          0   0   0
8:		         1   2  20   |          0   0   0
9:		        40  80 100   |          0   0   0

			read			write
offset                rpcs   % cum % |       rpcs   % cum %
0:		         1   2   2   |          0   0   0
1:		         0   0   2   |          0   0   0
2:		         0   0   2   |          0   0   0
4:		         0   0   2   |          0   0   0
8:		         0   0   2   |          0   0   0
16:		         0   0   2   |          0   0   0
32:		         0   0   2   |          0   0   0
64:		         0   0   2   |          0   0   0
128:		         0   0   2   |          0   0   0
256:		         1   2   4   |          0   0   0
512:		         2   4   8   |          0   0   0
1024:		         4   8  16   |          0   0   0
2048:		         8  16  32   |          0   0   0
4096:		        16  32  64   |          0   0   0
8192:		        18  36 100   |          0   0   0
osc.lustre-OST0001-osc-ffff8800a99f2000.rpc_stats=
snapshot_time:            1746139244.988360932 secs.nsecs
start_time:               1746138600.510640942 secs.nsecs
elapsed_time:             644.477719990 secs.nsecs
read RPCs in flight:  0
write RPCs in flight: 0
DIO RPCs in flight: 0
pending write pages:  0
pending read pages:   0

			read			write
pages per rpc         rpcs   % cum % |       rpcs   % cum %
1:		         3   0   0   |          3   1   1
2:		         0   0   0   |          0   0   1
4:		         0   0   0   |          0   0   1
8:		         0   0   0   |          0   0   1
16:		         0   0   0   |          0   0   1
32:		         0   0   0   |          0   0   1
64:		         0   0   0   |          0   0   1
128:		         0   0   0   |          0   0   1
256:		       352  99 100   |        150  98 100

			read			write
rpcs in flight        rpcs   % cum % |       rpcs   % cum %
1:		        37  10  10   |         17  11  11
2:		        27   7  18   |         89  58  69
3:		        19   5  23   |         38  24  94
4:		        18   5  28   |          8   5  99
5:		        13   3  32   |          1   0 100
6:		        13   3  35   |          0   0 100
7:		        13   3  39   |          0   0 100
8:		        20   5  45   |          0   0 100
9:		       195  54 100   |          0   0 100

			read			write
offset                rpcs   % cum % |       rpcs   % cum %
0:		        12   3   3   |          6   3   3
1:		         0   0   3   |          0   0   3
2:		         0   0   3   |          0   0   3
4:		         0   0   3   |          0   0   3
8:		         0   0   3   |          0   0   3
16:		         0   0   3   |          0   0   3
32:		         0   0   3   |          0   0   3
64:		         0   0   3   |          0   0   3
128:		         0   0   3   |          0   0   3
256:		         7   1   5   |          3   1   5
512:		        14   3   9   |          6   3   9
1024:		        28   7  17   |         12   7  17
2048:		        56  15  32   |         24  15  33
4096:		       112  31  64   |         48  31  64
8192:		       126  35 100   |         54  35 100
osc.lustre-OST0001-osc-ffff8800b6371000.rpc_stats=
snapshot_time:            1746139244.988430583 secs.nsecs
start_time:               1746138602.514919441 secs.nsecs
elapsed_time:             642.473511142 secs.nsecs
read RPCs in flight:  0
write RPCs in flight: 0
DIO RPCs in flight: 0
pending write pages:  0
pending read pages:   0

			read			write
pages per rpc         rpcs   % cum % |       rpcs   % cum %
1:		         0   0   0   |          0   0   0
2:		         0   0   0   |          0   0   0
4:		         0   0   0   |          0   0   0
8:		         0   0   0   |          0   0   0
16:		         0   0   0   |          0   0   0
32:		         0   0   0   |          0   0   0
64:		         0   0   0   |          0   0   0
128:		         0   0   0   |          0   0   0
256:		        50 100 100   |          0   0   0

			read			write
rpcs in flight        rpcs   % cum % |       rpcs   % cum %
1:		         2   4   4   |          0   0   0
2:		         1   2   6   |          0   0   0
3:		         1   2   8   |          0   0   0
4:		         1   2  10   |          0   0   0
5:		         1   2  12   |          0   0   0
6:		         1   2  14   |          0   0   0
7:		         2   4  18   |          0   0   0
8:		         2   4  22   |          0   0   0
9:		        39  78 100   |          0   0   0

			read			write
offset                rpcs   % cum % |       rpcs   % cum %
0:		         1   2   2   |          0   0   0
1:		         0   0   2   |          0   0   0
2:		         0   0   2   |          0   0   0
4:		         0   0   2   |          0   0   0
8:		         0   0   2   |          0   0   0
16:		         0   0   2   |          0   0   0
32:		         0   0   2   |          0   0   0
64:		         0   0   2   |          0   0   0
128:		         0   0   2   |          0   0   0
256:		         1   2   4   |          0   0   0
512:		         2   4   8   |          0   0   0
1024:		         4   8  16   |          0   0   0
2048:		         8  16  32   |          0   0   0
4096:		        16  32  64   |          0   0   0
8192:		        18  36 100   |          0   0   0
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d99b.sanity-pcc is idle now
PASS 99b (69s)

== sanity-pcc test 100: race among PCC unlink | read | write | detach for PCC-RO file ========================================================== 18:40:49 (1746139249)
60+0 records in
60+0 records out
62914560 bytes (63 MB) copied, 0.132068 s, 476 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d100.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/61440           done                            
Creating filesystem with 61440 1k blocks and 15360 inodes
Filesystem UUID: 6ad6997a-a1e3-4005-b4e7-8ce9f4f2893d
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345

Allocating group tables: 0/8   done                            
Writing inode tables: 0/8   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/8   done

/tmp/f100.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=6ad6997a-a1e3-4005-b4e7-8ce9f4f2893d (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={0}
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.796586 s, 65.8 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
Detach /mnt/lustre/f100.sanity-pcc ...
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
7+0 records in
7+0 records out
7340032 bytes (7.3 MB) copied, 0.265395 s, 27.7 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0371637 s, 28.2 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.856732 s, 61.2 MB/s
4+0 records in
4+0 records out
4194304 bytes (4.2 MB) copied, 0.261828 s, 16.0 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
2+0 records in
2+0 records out
2097152 bytes (2.1 MB) copied, 0.0511619 s, 41.0 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
Detach /mnt/lustre/f100.sanity-pcc ...
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.969057 s, 54.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.510304 s, 103 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
24+1 records in
24+1 records out
25956352 bytes (26 MB) copied, 0.156919 s, 165 MB/s
3+0 records in
3+0 records out
3145728 bytes (3.1 MB) copied, 0.0122551 s, 257 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
38+0 records in
38+0 records out
39845888 bytes (40 MB) copied, 0.130957 s, 304 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.810248 s, 64.7 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Detach /mnt/lustre/f100.sanity-pcc ...
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0473562 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00908174 s, 0.0 kB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
26+0 records in
26+0 records out
27262976 bytes (27 MB) copied, 0.0756251 s, 361 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.897025 s, 58.4 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.059415 s, 882 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00913697 s, 0.0 kB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.992765 s, 52.8 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.544116 s, 96.4 MB/s
2+0 records in
2+0 records out
2097152 bytes (2.1 MB) copied, 0.151167 s, 13.9 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0298251 s, 35.2 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.817027 s, 64.2 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.611784 s, 85.7 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
38+0 records in
38+0 records out
39845888 bytes (40 MB) copied, 0.507354 s, 78.5 MB/s
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.16748 s, 44.9 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0344569 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0170078 s, 0.0 kB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.28568 s, 40.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.917257 s, 57.2 MB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0849378 s, 0.0 kB/s
Detach /mnt/lustre/f100.sanity-pcc ...
14+0 records in
14+0 records out
14680064 bytes (15 MB) copied, 0.0444838 s, 330 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
37+0 records in
37+0 records out
38797312 bytes (39 MB) copied, 0.262012 s, 148 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0222164 s, 2.4 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.923366 s, 56.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0452736 s, 1.2 GB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.00481735 s, 0.0 kB/s
9+0 records in
9+0 records out
9437184 bytes (9.4 MB) copied, 0.0378769 s, 249 MB/s
14+0 records in
14+0 records out
14680064 bytes (15 MB) copied, 0.0394852 s, 372 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.972359 s, 53.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.594282 s, 88.2 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
8+0 records in
8+0 records out
8388608 bytes (8.4 MB) copied, 0.0409417 s, 205 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
40+0 records in
40+0 records out
41943040 bytes (42 MB) copied, 0.104751 s, 400 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.852448 s, 61.5 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.455516 s, 115 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0659277 s, 795 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.85093 s, 61.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.635113 s, 82.6 MB/s
43+1 records in
43+1 records out
45637632 bytes (46 MB) copied, 0.0443075 s, 1.0 GB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.407096 s, 129 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.904972 s, 57.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0568633 s, 922 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
Detach /mnt/lustre/f100.sanity-pcc ...
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.142077 s, 7.4 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
41+0 records in
41+0 records out
42991616 bytes (43 MB) copied, 0.20682 s, 208 MB/s
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
Detach /mnt/lustre/f100.sanity-pcc ...
dd: warning: partial read (286720 bytes); suggest iflag=fullblock
22+2 records in
22+2 records out
24117248 bytes (24 MB) copied, 0.252127 s, 95.7 MB/s
22+0 records in
22+0 records out
23068672 bytes (23 MB) copied, 0.175935 s, 131 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.04189 s, 50.3 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
Detach /mnt/lustre/f100.sanity-pcc ...
Detach /mnt/lustre/f100.sanity-pcc ...
30+0 records in
30+0 records out
31457280 bytes (31 MB) copied, 1.05839 s, 29.7 MB/s
3+0 records in
3+0 records out
3145728 bytes (3.1 MB) copied, 0.0767277 s, 41.0 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
Detach /mnt/lustre/f100.sanity-pcc ...
24+0 records in
24+0 records out
25165824 bytes (25 MB) copied, 0.0446637 s, 563 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.845453 s, 62.0 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
17+0 records in
17+0 records out
17825792 bytes (18 MB) copied, 0.445893 s, 40.0 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
2+0 records in
2+0 records out
2097152 bytes (2.1 MB) copied, 0.0143967 s, 146 MB/s
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.709576 s, 73.9 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
2+0 records in
2+0 records out
2097152 bytes (2.1 MB) copied, 0.0579641 s, 36.2 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0400726 s, 26.2 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.796279 s, 65.8 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.466021 s, 113 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0111486 s, 94.1 MB/s
44+0 records in
44+0 records out
46137344 bytes (46 MB) copied, 0.289204 s, 160 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0112728 s, 93.0 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00490059 s, 214 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.697862 s, 75.1 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
Detach /mnt/lustre/f100.sanity-pcc ...
38+0 records in
38+0 records out
39845888 bytes (40 MB) copied, 0.512955 s, 77.7 MB/s
dd: warning: partial read (290816 bytes); suggest iflag=fullblock
6+2 records in
6+2 records out
7340032 bytes (7.3 MB) copied, 0.0305192 s, 241 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.718481 s, 73.0 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.491035 s, 107 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.152757 s, 343 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.896342 s, 58.5 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.558155 s, 93.9 MB/s
39+0 records in
39+0 records out
40894464 bytes (41 MB) copied, 0.337538 s, 121 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.929773 s, 56.4 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
Detach /mnt/lustre/f100.sanity-pcc ...
14+0 records in
14+0 records out
14680064 bytes (15 MB) copied, 0.461672 s, 31.8 MB/s
5+0 records in
5+0 records out
5242880 bytes (5.2 MB) copied, 0.0279074 s, 188 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.10844 s, 47.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.975573 s, 53.7 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.168683 s, 0.0 kB/s
6+0 records in
6+0 records out
6291456 bytes (6.3 MB) copied, 0.0475396 s, 132 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.962291 s, 54.5 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.620971 s, 84.4 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Detach /mnt/lustre/f100.sanity-pcc ...
15+0 records in
15+0 records out
15728640 bytes (16 MB) copied, 0.171119 s, 91.9 MB/s
31+0 records in
31+0 records out
32505856 bytes (33 MB) copied, 0.0742403 s, 438 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.12507 s, 46.6 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.254481 s, 206 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
6+0 records in
6+0 records out
6291456 bytes (6.3 MB) copied, 0.0274868 s, 229 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
Detach /mnt/lustre/f100.sanity-pcc ...
37+0 records in
37+0 records out
38797312 bytes (39 MB) copied, 0.274546 s, 141 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.733106 s, 71.5 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.211242 s, 248 MB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0990765 s, 0.0 kB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
Detach /mnt/lustre/f100.sanity-pcc ...
42+0 records in
42+0 records out
44040192 bytes (44 MB) copied, 0.516055 s, 85.3 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.985259 s, 53.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0746653 s, 702 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0228765 s, 2.3 GB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
Detach /mnt/lustre/f100.sanity-pcc ...
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.983135 s, 53.3 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.537937 s, 97.5 MB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0974056 s, 0.0 kB/s
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.128693 s, 97.8 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.04595 s, 50.1 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.686548 s, 76.4 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0321139 s, 32.7 MB/s
43+0 records in
43+0 records out
Detach /mnt/lustre/f100.sanity-pcc ...
45088768 bytes (45 MB) copied, 0.456882 s, 98.7 MB/s
34+0 records in
34+0 records out
35651584 bytes (36 MB) copied, 0.229847 s, 155 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
Detach /mnt/lustre/f100.sanity-pcc ...
48+0 records in
48+0 records out
50331648 bytes (50 MB) copied, 0.0553391 s, 910 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.08856 s, 48.2 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.706408 s, 74.2 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
30+0 records in
30+0 records out
31457280 bytes (31 MB) copied, 0.218959 s, 144 MB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.01992 s, 0.0 kB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.696909 s, 75.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.374114 s, 140 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.09178 s, 48.0 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0109514 s, 0.0 kB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0147676 s, 0.0 kB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.706388 s, 74.2 MB/s
3+0 records in
3+0 records out
3145728 bytes (3.1 MB) copied, 0.103198 s, 30.5 MB/s
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0183403 s, 57.2 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.901496 s, 58.2 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0882589 s, 11.9 MB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0169333 s, 0.0 kB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.760641 s, 68.9 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.306572 s, 171 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0260316 s, 2.0 GB/s
41+0 records in
41+0 records out
42991616 bytes (43 MB) copied, 0.206975 s, 208 MB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0197938 s, 0.0 kB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
Detach /mnt/lustre/f100.sanity-pcc ...
35+0 records in
35+0 records out
36700160 bytes (37 MB) copied, 0.237199 s, 155 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 1.02586 s, 51.1 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0683374 s, 767 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0377674 s, 0.0 kB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
22+0 records in
22+0 records out
23068672 bytes (23 MB) copied, 0.182986 s, 126 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
37+0 records in
37+0 records out
38797312 bytes (39 MB) copied, 0.035497 s, 1.1 GB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.941804 s, 55.7 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
unlink: cannot unlink '/mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0': No such file or directory
2+0 records in
2+0 records out
2097152 bytes (2.1 MB) copied, 0.125943 s, 16.7 MB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0208036 s, 0.0 kB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.456279 s, 115 MB/s
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.842812 s, 62.2 MB/s
Detach /mnt/lustre/f100.sanity-pcc ...
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0713127 s, 735 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0293313 s, 1.8 GB/s
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.0116313 s, 0.0 kB/s
Detach /mnt/lustre/f100.sanity-pcc ...
Unlink /mnt/pcc.d100.sanity-pcc/d100.sanity-pcc/59b1/0x2000059f1:0x40:0x0
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.939406 s, 55.8 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.556623 s, 94.2 MB/s
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.0370089 s, 1.4 GB/s
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d100.sanity-pcc is idle now
PASS 100 (68s)

== sanity-pcc test 101a: Test auto attach in mount namespace (simulated container) ========================================================== 18:41:57 (1746139317)
creating user namespace for 500
sanityu+ 10140  0.0  0.0 108060   364 pts/0    S+   18:41   0:00 sleep 600
root     10142  0.0  0.0 114880   984 pts/0    S+   18:41   0:00 grep sleep
Created NS: child (sleep) pid=10140 in 0 seconds
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [newuidmap] [10140] [0] [500] [1]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [newgidmap] [10140] [0] [500] [1]
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.12625 s, 415 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d101a.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: d1b18606-0dfa-4f3c-ae89-cedcb5eef6e9
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f101a.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=d1b18606-0dfa-4f3c-ae89-cedcb5eef6e9 (extents) (64bit) (large files) (huge files)
Check for temp file in PCC mount
Check for temp file in PCC mount from inside namespace
Starting copytool 'agt1' on 'oleg101-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/mnt/pcc.d101a.sanity-pcc/d101a.sanity-pcc --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2  "/mnt/lustre"'
Verify open attach from inside mount namespace
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00907559 s, 113 kB/s
file: /mnt/lustre/d101a.sanity-pcc/f101a.sanity-pcc, type: readwrite, PCC_file: /d101a.sanity-pcc/59a1/0x2000059f1:0x50:0x0, open_count: 0, flags: 0
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [state] [/mnt/lustre/d101a.sanity-pcc/f101a.sanity-pcc]
ldlm.namespaces.lustre-MDT0000-mdc-ffff8800a99f2000.lru_size=clear
ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b6371000.lru_size=clear
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [state] [/mnt/lustre/d101a.sanity-pcc/f101a.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [multiop] [/mnt/lustre/d101a.sanity-pcc/f101a.sanity-pcc] [oc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [state] [/mnt/lustre/d101a.sanity-pcc/f101a.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [state] [/mnt/lustre/d101a.sanity-pcc/f101a.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [state] [/mnt/lustre/d101a.sanity-pcc/f101a.sanity-pcc]
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00100947 s, 1.0 MB/s
Verify auto attach at open from inside NS for RW-PCC
ldlm.namespaces.lustre-MDT0000-mdc-ffff8800a99f2000.lru_size=clear
ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b6371000.lru_size=clear
Waiting 200s for 'SUCCEED'
Updated after 2s: want 'SUCCEED' got 'SUCCEED'
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d101a.sanity-pcc is idle now
PASS 101a (12s)

== sanity-pcc test 102: PCC-RO should not hange for io_uring I/O engine ========================================================== 18:42:09 (1746139329)

 SKIP: sanity-pcc test_102 Client OS does not support io_uring I/O engine
SKIP 102 (3s)

== sanity-pcc test 203: Verify attach/hit bytes statistics data ========================================================== 18:42:12 (1746139332)
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.0301897 s, 347 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d203.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/10240           done                            
Creating filesystem with 10240 1k blocks and 2560 inodes
Filesystem UUID: f1f8b239-028f-4d60-96b1-132db30d0a32
Superblock backups stored on blocks: 
	8193

Allocating group tables: 0/2   done                            
Writing inode tables: 0/2   done                            
Creating journal (2048 blocks): done
Writing superblocks and filesystem accounting information: 0/2   done

/tmp/f203.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=f1f8b239-028f-4d60-96b1-132db30d0a32 (extents) (64bit) (large files) (huge files)
pcc:
  -
    pccpath: /mnt/pcc.d203.sanity-pcc/d203.sanity-pcc
    hsmtool: posix_v2
    rwid: 0
    roid: 2
    flags: ae
    autocache: projid={0}
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00544336 s, 188 kB/s
==== stats ====
llite.lustre-ffff8800a99f2000.stats=
snapshot_time             1746139334.359038735 secs.nsecs
start_time                1746139334.273280165 secs.nsecs
elapsed_time              0.085758570 secs.nsecs
read_bytes                1 samples [bytes] 1024 1024 1024 1048576
write_bytes               1 samples [bytes] 1024 1024 1024 1048576
read                      3 samples [usecs] 136 5397 5671 29165149
write                     1 samples [usecs] 3131 3131 3131 9803161
open                      3 samples [usecs] 19 18864 18908 355851482
close                     3 samples [usecs] 1903 4844 11491 49591281
getattr                   2 samples [usecs] 79 3624 3703 13139617
mknod                     1 samples [usecs] 9271 9271 9271 85951441
inode_permission          7 samples [usecs] 2 3459 3714 11985204
opencount                 3 samples [reqs] 1 3 6 14
openclosetime             2 samples [usecs] 10740 18343 29083 451813249
pcc_attach                1 samples [reqs]
pcc_hit_bytes             2 samples [bytes] 1024 1024 2048 2097152
pcc_attach_bytes          1 samples [bytes] 1024 1024 1024 1048576
llite.lustre-ffff8800b6371000.stats=
snapshot_time             1746139334.360761875 secs.nsecs
start_time                1746139334.273301986 secs.nsecs
elapsed_time              0.087459889 secs.nsecs
attach_num: 1, attach_bytes: 1024, hit_num: 2, hit_bytes: 2048
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d203.sanity-pcc is idle now
PASS 203 (6s)

== sanity-pcc test 204a: pin/unpin pcc flag ============== 18:42:18 (1746139338)
50+0 records in
50+0 records out
52428800 bytes (52 MB) copied, 0.110246 s, 476 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d204a.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/51200           done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: f4d9ddce-8af3-4665-b917-c1940ca3cf84
Superblock backups stored on blocks: 
	8193, 24577, 40961

Allocating group tables: 0/7   done                            
Writing inode tables: 0/7   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/7   done

/tmp/f204a.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=f4d9ddce-8af3-4665-b917-c1940ca3cf84 (extents) (64bit) (large files) (huge files)
getfattr: Removing leading '/' from absolute path names
# file: mnt/lustre/f204a.sanity-pcc.dat
trusted.pin="[hsm: 2]"

getfattr: Removing leading '/' from absolute path names
getfattr: Removing leading '/' from absolute path names
# file: mnt/lustre/f204a.sanity-pcc.dat
trusted.pin="[hsm: 2,hsm: 100]"

getfattr: Removing leading '/' from absolute path names
getfattr: Removing leading '/' from absolute path names
# file: mnt/lustre/f204a.sanity-pcc.dat
trusted.pin="[hsm: 2,hsm: 100]"

getfattr: Removing leading '/' from absolute path names
# file: mnt/lustre/f204a.sanity-pcc.dat
trusted.pin="[hsm: 2]"

getfattr: Removing leading '/' from absolute path names
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d204a.sanity-pcc is idle now
PASS 204a (5s)

== sanity-pcc test 204b: Permission check for the pin/unpin operation ========================================================== 18:42:24 (1746139344)
mdt.lustre-MDT0000.enable_pin_gid=0
/home/green/git/lustre-release/lustre/tests/sanity-pcc.sh: illegal option -- p
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [touch] [/mnt/lustre/d204b.sanity-pcc/f204b.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [pin] [-i] [100] [/mnt/lustre/d204b.sanity-pcc/f204b.sanity-pcc]
pin: cannot pin '/mnt/lustre/d204b.sanity-pcc/f204b.sanity-pcc' for PCC: Operation not permitted
getfattr: Removing leading '/' from absolute path names
# file: mnt/lustre/d204b.sanity-pcc/f204b.sanity-pcc
trusted.pin="[hsm: 100]"

running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [unpin] [-i] [100] [/mnt/lustre/d204b.sanity-pcc/f204b.sanity-pcc]
unpin: cannot unpin '/mnt/lustre/d204b.sanity-pcc/f204b.sanity-pcc' for PCC: Operation not permitted
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [pin] [-i] [100] [/mnt/lustre/d204b.sanity-pcc/f204b.sanity-pcc]
pin: cannot pin '/mnt/lustre/d204b.sanity-pcc/f204b.sanity-pcc' for PCC: Operation not permitted
mdt.lustre-MDT0000.enable_pin_gid=-1
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [pin] [-i] [100] [/mnt/lustre/d204b.sanity-pcc/f204b.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [unpin] [-i] [100] [/mnt/lustre/d204b.sanity-pcc/f204b.sanity-pcc]
mdt.lustre-MDT0000.enable_pin_gid=500
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [pin] [-i] [100] [/mnt/lustre/d204b.sanity-pcc/f204b.sanity-pcc]
running as uid/gid/euid/egid 500/500/500/500, groups: 500
 [/home/green/git/lustre-release/lustre/utils/lfs] [pcc] [unpin] [-i] [100] [/mnt/lustre/d204b.sanity-pcc/f204b.sanity-pcc]
mdt.lustre-MDT0000.enable_pin_gid=0
PASS 204b (6s)

== sanity-pcc test 204c: PCC pin/unpin without attach ID specified ========================================================== 18:42:30 (1746139350)
pin: failed to get id for '/mnt/lustre/f204c.sanity-pcc': No such file or directory
60+0 records in
60+0 records out
62914560 bytes (63 MB) copied, 0.146477 s, 430 MB/s
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
umount: /mnt/pcc.d204c.sanity-pcc: not mounted
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_prio,net_cls)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/nbd0 on / type ext4 (ro,relatime,stripe=32,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13087)
mqueue on /dev/mqueue type mqueue (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
none on /mnt type ramfs (rw,relatime)
none on /var/lib/stateless/writable type tmpfs (rw,relatime)
/dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime)
none on /var/cache/man type tmpfs (rw,relatime)
none on /var/log type tmpfs (rw,relatime)
none on /var/lib/dbus type tmpfs (rw,relatime)
none on /tmp type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
none on /var/tmp type tmpfs (rw,relatime)
none on /var/lib/NetworkManager type tmpfs (rw,relatime)
none on /var/lib/systemd/random-seed type tmpfs (rw,relatime)
none on /var/spool type tmpfs (rw,relatime)
none on /var/lib/nfs type tmpfs (rw,relatime)
none on /var/lib/gssproxy type tmpfs (rw,relatime)
none on /var/lib/logrotate type tmpfs (rw,relatime)
none on /etc type tmpfs (rw,relatime)
none on /var/lib/rsyslog type tmpfs (rw,relatime)
none on /var/lib/dhclient type tmpfs (rw,relatime)
192.168.200.253:/exports/state/oleg101-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
192.168.200.253:/exports/state/oleg101-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
192.168.200.253:/exports/testreports/51284/testresults/sanity-pcc-zfs-centos7_x86_64-centos7_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.1,local_lock=none,addr=192.168.200.253)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=382032k,mode=700)
/dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime)
192.168.201.101@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
192.168.201.101@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose)
mke2fs 1.46.2.wc5 (26-Mar-2022)
Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.
Discarding device blocks:  1024/61440           done                            
Creating filesystem with 61440 1k blocks and 15360 inodes
Filesystem UUID: 0c6c80d2-58e8-4a0c-9d1c-0693b66661b0
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345

Allocating group tables: 0/8   done                            
Writing inode tables: 0/8   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/8   done

/tmp/f204c.sanity-pcc: Linux rev 1.0 ext4 filesystem data, UUID=0c6c80d2-58e8-4a0c-9d1c-0693b66661b0 (extents) (64bit) (large files) (huge files)
getfattr: Removing leading '/' from absolute path names
# file: mnt/lustre/f204c.sanity-pcc
trusted.pin="[hsm: 2]"

getfattr: Removing leading '/' from absolute path names
/mnt/lustre/f204c.sanity-pcc: trusted.pin: No such attribute
/mnt/lustre/f204c.sanity-pcc: trusted.pin: No such attribute
Cleanup PCC backend on /mnt/lustre
/mnt/pcc.d204c.sanity-pcc is idle now
PASS 204c (5s)
== sanity-pcc test complete, duration 2220 sec =========== 18:42:35 (1746139355)
sanity-pcc: FAIL: test_1d expected /mnt/pcc.d1d.sanity-pcc/d1d.sanity-pcc/13a3/0x2000013a1:0x2:0x0 size: 7654321 got: 0
=== sanity-pcc: start cleanup 18:42:36 (1746139356) ===
Stopping clients: oleg101-client.virtnet /mnt/lustre2 (opts:)
Stopping client oleg101-client.virtnet /mnt/lustre2 opts:
=== sanity-pcc: finish cleanup 18:42:42 (1746139362) ===