== sanity test 398g: verify parallel dio async RPC submission ========================================================== 03:24:15 (1786173855) 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.982569 s, 8.5 MB/s osc.lustre-OST0000-osc-ffff9fa91d8d7800.max_pages_per_rpc=1M fail_loc=0x214 fail_val=2 osc.lustre-OST0000-osc-ffff9fa91d8d7800.rpc_stats=c osc.lustre-OST0001-osc-ffff9fa91d8d7800.rpc_stats=c 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 2.67004 s, 3.1 MB/s osc.lustre-OST0000-osc-ffff9fa91d8d7800.rpc_stats= snapshot_time: 1786173862.239667504 secs.nsecs start_time: 1786173859.421719836 secs.nsecs elapsed_time: 2.817947668 secs.nsecs read RPCs in flight: 0 write RPCs in flight: 0 DIO RPCs in flight: 0 pending write pages: 0 pending read pages: 0 read write pages per rpc rpcs % cum % | rpcs % cum % 1: 0 0 0 | 0 0 0 2: 0 0 0 | 0 0 0 4: 0 0 0 | 0 0 0 8: 0 0 0 | 0 0 0 16: 0 0 0 | 0 0 0 32: 0 0 0 | 0 0 0 64: 0 0 0 | 0 0 0 128: 0 0 0 | 0 0 0 256: 0 0 0 | 8 100 100 read write rpcs in flight rpcs % cum % | rpcs % cum % 1: 0 0 0 | 0 0 0 2: 0 0 0 | 1 12 12 3: 0 0 0 | 1 12 25 4: 0 0 0 | 1 12 37 5: 0 0 0 | 1 12 50 6: 0 0 0 | 1 12 62 7: 0 0 0 | 1 12 75 8: 0 0 0 | 1 12 87 9: 0 0 0 | 1 12 100 read write offset rpcs % cum % | rpcs % cum % 0: 0 0 0 | 2 25 25 1: 0 0 0 | 0 0 25 2: 0 0 0 | 0 0 25 4: 0 0 0 | 0 0 25 8: 0 0 0 | 0 0 25 16: 0 0 0 | 0 0 25 32: 0 0 0 | 0 0 25 64: 0 0 0 | 0 0 25 128: 0 0 0 | 0 0 25 256: 0 0 0 | 2 25 50 512: 0 0 0 | 4 50 100 read write RPC latency (us) count % cum % | count % cum % 0: 0 0 0 | 0 0 0 1: 0 0 0 | 0 0 0 2: 0 0 0 | 0 0 0 4: 0 0 0 | 0 0 0 8: 0 0 0 | 0 0 0 16: 0 0 0 | 0 0 0 32: 0 0 0 | 0 0 0 64: 0 0 0 | 0 0 0 128: 0 0 0 | 0 0 0 256: 0 0 0 | 0 0 0 512: 0 0 0 | 0 0 0 1024: 0 0 0 | 0 0 0 2048: 0 0 0 | 0 0 0 4096: 0 0 0 | 0 0 0 8192: 0 0 0 | 0 0 0 16384: 0 0 0 | 0 0 0 32768: 0 0 0 | 0 0 0 65536: 0 0 0 | 0 0 0 131072: 0 0 0 | 0 0 0 262144: 0 0 0 | 0 0 0 524288: 0 0 0 | 0 0 0 1048576: 0 0 0 | 1 11 11 2097152: 0 0 0 | 8 88 100 sanity test_398g: @@@@@@ FAIL: we should see 8 RPCs in flight Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7496:error() = /home/green/git/lustre-release/lustre/tests/sanity.sh:31580:test_398g() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7873:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7936:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7735:run_test() = /home/green/git/lustre-release/lustre/tests/sanity.sh:31603:main() Dumping lctl log to /tmp/testlogs//sanity.test_398g.*.1786173864.log rsync: chown "/tmp/testlogs/.sanity.test_398g.debug_log.oleg319-server.1786173864.log.9fefQ6" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanity.test_398g.dmesg.oleg319-server.1786173864.log.DuOisI" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] fail_loc=0