== sanity test 123ad: Verify batching statahead works correctly ========================================================== 22:47:16 (1780454836) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff961089393000.batch_stats=0 llite.lustre-ffff961089393000.statahead_max=32 llite.lustre-ffff961089393000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff961089393000.batch_stats=0 total: 100 open/close in 2.53 seconds: 39.58 ops/second llite.lustre-ffff961089393000.statahead_max=0 101 real 0m2.240s user 0m0.007s sys 0m0.330s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff961089393000.statahead_max=32 llite.lustre-ffff961089393000.statahead_max=32 101 real 0m1.155s user 0m0.003s sys 0m0.172s 'ls -l' 100 files with statahead: 1 sec statahead total: 138 statahead wrong: 2 agl total: 132 list_total: 31 fname_total: 95 hit_total: 4099 miss_total: 1616 swrong=2 ewrong=2 - open/close 797 (time 1780454862.03 total 10.01 last 79.58) total: 900 open/close in 11.73 seconds: 76.75 ops/second llite.lustre-ffff961089393000.statahead_max=0 1001 real 0m21.024s user 0m0.033s sys 0m3.092s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff961089393000.statahead_max=32 llite.lustre-ffff961089393000.statahead_max=32 1001 real 0m8.806s user 0m0.017s sys 0m1.535s 'ls -l' 1000 files with statahead: 9 sec statahead total: 139 statahead wrong: 2 agl total: 133 list_total: 32 fname_total: 95 hit_total: 5096 miss_total: 1619 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff961089393000.statahead_stats= statahead total: 139 statahead wrong: 2 agl total: 133 list_total: 32 fname_total: 95 hit_total: 5096 miss_total: 1619 mdc.lustre-MDT0000-mdc-ffff961089393000.batch_stats= snapshot_time: 1780454928.974912336 secs.nsecs start_time: 1780454840.426983955 secs.nsecs elapsed_time: 88.547928381 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 663 (time 1780454942.06 total 10.01 last 66.25) - open/close 1370 (time 1780454952.07 total 20.01 last 70.65) - open/close 2132 (time 1780454962.08 total 30.02 last 76.15) - open/close 2927 (time 1780454972.08 total 40.02 last 79.48) - open/close 3710 (time 1780454982.10 total 50.04 last 78.15) - open/close 4474 (time 1780454992.12 total 60.06 last 76.29) - open/close 5178 (time 1780455002.12 total 70.06 last 70.37) - open/close 5940 (time 1780455012.13 total 80.07 last 76.13) - open/close 6691 (time 1780455022.13 total 90.07 last 75.08) - open/close 7472 (time 1780455032.13 total 100.08 last 78.08) - open/close 8068 (time 1780455042.13 total 110.08 last 59.59) - open/close 8757 (time 1780455052.20 total 120.14 last 68.48) - open/close 9537 (time 1780455062.20 total 130.15 last 77.94) total: 10000 open/close in 136.30 seconds: 73.37 ops/second llite.lustre-ffff961089393000.statahead_batch_max=0 llite.lustre-ffff961089393000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff961089393000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.521s user 0m0.307s sys 0m16.346s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff961089393000.statahead_batch_max=32 llite.lustre-ffff961089393000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff961089393000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff961089393000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m37.124s user 0m0.282s sys 0m15.363s batched statahead RPC= hit count=9974 mdc.lustre-MDT0000-mdc-ffff961089393000.batch_stats= snapshot_time: 1780455247.030974020 secs.nsecs start_time: 1780455136.687520461 secs.nsecs elapsed_time: 110.343453559 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 1 32: 313 98 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff961089393000.batch_stats=0 llite.lustre-ffff961089393000.statahead_max=2048 llite.lustre-ffff961089393000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff961089393000.batch_stats=0 total: 100 open/close in 3.31 seconds: 30.17 ops/second llite.lustre-ffff961089393000.statahead_max=0 101 real 0m3.099s user 0m0.006s sys 0m0.335s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff961089393000.statahead_max=2048 llite.lustre-ffff961089393000.statahead_max=2048 101 real 0m1.074s user 0m0.004s sys 0m0.160s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10073 miss_total: 27 swrong=0 ewrong=0 - open/close 583 (time 1780455570.33 total 10.02 last 58.17) total: 900 open/close in 14.66 seconds: 61.40 ops/second llite.lustre-ffff961089393000.statahead_max=0 1001 real 0m21.277s user 0m0.029s sys 0m3.112s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff961089393000.statahead_max=2048 llite.lustre-ffff961089393000.statahead_max=2048 1001 real 0m9.345s user 0m0.023s sys 0m1.468s 'ls -l' 1000 files with statahead: 10 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff961089393000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 mdc.lustre-MDT0000-mdc-ffff961089393000.batch_stats= snapshot_time: 1780455641.884821369 secs.nsecs start_time: 1780455249.505775733 secs.nsecs elapsed_time: 392.379045636 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 739 (time 1780455654.47 total 10.01 last 73.82) - open/close 1485 (time 1780455664.49 total 20.03 last 74.48) - open/close 2215 (time 1780455674.49 total 30.03 last 72.96) - open/close 2945 (time 1780455684.51 total 40.04 last 72.90) - open/close 3769 (time 1780455694.52 total 50.06 last 82.31) - open/close 4596 (time 1780455704.54 total 60.08 last 82.53) - open/close 5341 (time 1780455714.55 total 70.09 last 74.38) - open/close 6063 (time 1780455724.56 total 80.10 last 72.13) - open/close 6809 (time 1780455734.57 total 90.10 last 74.58) - open/close 7501 (time 1780455744.57 total 100.11 last 69.20) - open/close 8170 (time 1780455754.57 total 110.11 last 66.90) - open/close 8872 (time 1780455764.57 total 120.11 last 70.17) - open/close 9629 (time 1780455774.58 total 130.12 last 75.66) total: 10000 open/close in 134.94 seconds: 74.11 ops/second llite.lustre-ffff961089393000.statahead_batch_max=0 llite.lustre-ffff961089393000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff961089393000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.596s user 0m0.303s sys 0m16.072s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff961089393000.statahead_batch_max=256 llite.lustre-ffff961089393000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff961089393000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff961089393000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.789s user 0m0.298s sys 0m16.352s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff961089393000.batch_stats= snapshot_time: 1780455931.258967542 secs.nsecs start_time: 1780455857.480902022 secs.nsecs elapsed_time: 73.778065520 secs.nsecs subreqs per batch batches % cum % 1: 89 37 37 2: 36 15 52 4: 26 10 63 8: 18 7 71 16: 12 5 76 32: 12 5 81 64: 8 3 84 128: 1 0 85 256: 35 14 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff961089393000.statahead_batch_max=64 llite.lustre-ffff961089393000.statahead_max=128