== sanity test 398g: verify parallel dio async RPC submission ========================================================== 23:04:58 (1789268698) 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.727654 s, 11.5 MB/s osc.lustre-OST0000-osc-ffff9742c2b62800.max_pages_per_rpc=1M fail_loc=0x214 fail_val=2 osc.lustre-OST0000-osc-ffff9742c2b62800.rpc_stats=c osc.lustre-OST0001-osc-ffff9742c2b62800.rpc_stats=c 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 2.57033 s, 3.3 MB/s osc.lustre-OST0000-osc-ffff9742c2b62800.rpc_stats= snapshot_time: 1789268706.665196796 secs.nsecs start_time: 1789268703.874121064 secs.nsecs elapsed_time: 2.791075732 secs.nsecs read RPCs in flight: 0 write RPCs in flight: 0 DIO RPCs in flight: 0 pending write pages: 0 pending read pages: 0 read write pages per rpc rpcs % cum % | rpcs % cum % 1: 0 0 0 | 0 0 0 2: 0 0 0 | 0 0 0 4: 0 0 0 | 0 0 0 8: 0 0 0 | 0 0 0 16: 0 0 0 | 0 0 0 32: 0 0 0 | 0 0 0 64: 0 0 0 | 0 0 0 128: 0 0 0 | 0 0 0 256: 0 0 0 | 8 100 100 read write rpcs in flight rpcs % cum % | rpcs % cum % 1: 0 0 0 | 0 0 0 2: 0 0 0 | 1 12 12 3: 0 0 0 | 1 12 25 4: 0 0 0 | 1 12 37 5: 0 0 0 | 1 12 50 6: 0 0 0 | 1 12 62 7: 0 0 0 | 1 12 75 8: 0 0 0 | 1 12 87 9: 0 0 0 | 1 12 100 read write offset rpcs % cum % | rpcs % cum % 0: 0 0 0 | 2 25 25 1: 0 0 0 | 0 0 25 2: 0 0 0 | 0 0 25 4: 0 0 0 | 0 0 25 8: 0 0 0 | 0 0 25 16: 0 0 0 | 0 0 25 32: 0 0 0 | 0 0 25 64: 0 0 0 | 0 0 25 128: 0 0 0 | 0 0 25 256: 0 0 0 | 2 25 50 512: 0 0 0 | 4 50 100 read write RPC latency (us) count % cum % | count % cum % 0: 0 0 0 | 0 0 0 1: 0 0 0 | 0 0 0 2: 0 0 0 | 0 0 0 4: 0 0 0 | 0 0 0 8: 0 0 0 | 0 0 0 16: 0 0 0 | 0 0 0 32: 0 0 0 | 0 0 0 64: 0 0 0 | 0 0 0 128: 0 0 0 | 0 0 0 256: 0 0 0 | 0 0 0 512: 0 0 0 | 0 0 0 1024: 0 0 0 | 0 0 0 2048: 0 0 0 | 0 0 0 4096: 0 0 0 | 0 0 0 8192: 0 0 0 | 0 0 0 16384: 0 0 0 | 0 0 0 32768: 0 0 0 | 0 0 0 65536: 0 0 0 | 0 0 0 131072: 0 0 0 | 0 0 0 262144: 0 0 0 | 0 0 0 524288: 0 0 0 | 0 0 0 1048576: 0 0 0 | 1 11 11 2097152: 0 0 0 | 8 88 100 sanity test_398g: @@@@@@ FAIL: we should see 8 RPCs in flight Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7524:error() = /home/green/git/lustre-release/lustre/tests/sanity.sh:31910:test_398g() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7901:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7964:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7763:run_test() = /home/green/git/lustre-release/lustre/tests/sanity.sh:31933:main() Dumping lctl log to /tmp/testlogs//sanity.test_398g.*.1789268708.log rsync: chown "/tmp/testlogs/.sanity.test_398g.debug_log.oleg105-server.1789268708.log.4eBnXe" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanity.test_398g.dmesg.oleg105-server.1789268708.log.E7cKXJ" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] fail_loc=0