== sanity test 123ad: Verify batching statahead works correctly ========================================================== 08:29:01 (1787833741) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9b94714d1000.batch_stats=0 llite.lustre-ffff9b94714d1000.statahead_max=32 llite.lustre-ffff9b94714d1000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9b94714d1000.batch_stats=0 total: 100 open/close in 2.38 seconds: 41.93 ops/second llite.lustre-ffff9b94714d1000.statahead_max=0 101 real 0m2.390s user 0m0.003s sys 0m0.316s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9b94714d1000.statahead_max=32 llite.lustre-ffff9b94714d1000.statahead_max=32 101 real 0m1.028s user 0m0.004s sys 0m0.169s 'ls -l' 100 files with statahead: 1 sec statahead total: 129 statahead wrong: 1 agl total: 123 list_total: 29 fname_total: 99 hit_total: 3902 miss_total: 248 swrong=1 ewrong=1 - open/close 845 (time 1787833767.44 total 10.02 last 84.30) total: 900 open/close in 10.88 seconds: 82.73 ops/second llite.lustre-ffff9b94714d1000.statahead_max=0 1001 real 0m17.218s user 0m0.030s sys 0m2.716s 'ls -l' 1000 files without statahead: 17 sec llite.lustre-ffff9b94714d1000.statahead_max=32 llite.lustre-ffff9b94714d1000.statahead_max=32 1001 real 0m10.018s user 0m0.019s sys 0m1.248s 'ls -l' 1000 files with statahead: 10 sec statahead total: 130 statahead wrong: 1 agl total: 124 list_total: 30 fname_total: 99 hit_total: 4901 miss_total: 249 swrong=1 ewrong=1 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff9b94714d1000.statahead_stats= statahead total: 130 statahead wrong: 1 agl total: 124 list_total: 30 fname_total: 99 hit_total: 4901 miss_total: 249 mdc.lustre-MDT0000-mdc-ffff9b94714d1000.batch_stats= snapshot_time: 1787833829.002823468 secs.nsecs start_time: 1787833745.940826711 secs.nsecs elapsed_time: 83.061996757 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 799 (time 1787833841.61 total 10.01 last 79.85) - open/close 1646 (time 1787833851.62 total 20.02 last 84.60) - open/close 2443 (time 1787833861.63 total 30.02 last 79.64) - open/close 3304 (time 1787833871.63 total 40.03 last 86.08) - open/close 4077 (time 1787833881.64 total 50.04 last 77.23) - open/close 4925 (time 1787833891.64 total 60.04 last 84.76) - open/close 5736 (time 1787833901.64 total 70.04 last 81.09) - open/close 6696 (time 1787833911.65 total 80.05 last 95.90) - open/close 7549 (time 1787833921.66 total 90.06 last 85.26) - open/close 8343 (time 1787833931.67 total 100.07 last 79.33) - open/close 8983 (time 1787833941.68 total 110.07 last 63.94) - open/close 9615 (time 1787833951.68 total 120.08 last 63.19) total: 10000 open/close in 125.05 seconds: 79.97 ops/second llite.lustre-ffff9b94714d1000.statahead_batch_max=0 llite.lustre-ffff9b94714d1000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b94714d1000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.325s user 0m0.288s sys 0m12.887s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9b94714d1000.statahead_batch_max=32 llite.lustre-ffff9b94714d1000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b94714d1000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9b94714d1000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m28.592s user 0m0.286s sys 0m12.598s batched statahead RPC= hit count=9981 mdc.lustre-MDT0000-mdc-ffff9b94714d1000.batch_stats= snapshot_time: 1787834124.543428042 secs.nsecs start_time: 1787834025.849778819 secs.nsecs elapsed_time: 98.693649223 secs.nsecs subreqs per batch batches %% cum %% 1: 2 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 1 32: 313 98 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9b94714d1000.batch_stats=0 llite.lustre-ffff9b94714d1000.statahead_max=2048 llite.lustre-ffff9b94714d1000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9b94714d1000.batch_stats=0 total: 100 open/close in 2.54 seconds: 39.33 ops/second llite.lustre-ffff9b94714d1000.statahead_max=0 101 real 0m1.919s user 0m0.007s sys 0m0.305s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9b94714d1000.statahead_max=2048 llite.lustre-ffff9b94714d1000.statahead_max=2048 101 real 0m0.981s user 0m0.004s sys 0m0.142s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10080 miss_total: 20 swrong=0 ewrong=0 - open/close 641 (time 1787834416.71 total 10.00 last 64.07) total: 900 open/close in 13.37 seconds: 67.30 ops/second llite.lustre-ffff9b94714d1000.statahead_max=0 1001 real 0m20.102s user 0m0.029s sys 0m2.878s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff9b94714d1000.statahead_max=2048 llite.lustre-ffff9b94714d1000.statahead_max=2048 1001 real 0m5.076s user 0m0.026s sys 0m1.172s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff9b94714d1000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 mdc.lustre-MDT0000-mdc-ffff9b94714d1000.batch_stats= snapshot_time: 1787834481.230026019 secs.nsecs start_time: 1787834126.727148351 secs.nsecs elapsed_time: 354.502877668 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 704 (time 1787834493.60 total 10.01 last 70.33) - open/close 1468 (time 1787834503.60 total 20.01 last 76.38) - open/close 2341 (time 1787834513.61 total 30.02 last 87.23) - open/close 3057 (time 1787834523.62 total 40.03 last 71.51) - open/close 3708 (time 1787834533.64 total 50.05 last 65.03) - open/close 4473 (time 1787834543.64 total 60.05 last 76.48) - open/close 5282 (time 1787834553.64 total 70.05 last 80.89) - open/close 6192 (time 1787834563.64 total 80.05 last 90.98) - open/close 7025 (time 1787834573.65 total 90.06 last 83.20) - open/close 7793 (time 1787834583.66 total 100.07 last 76.75) - open/close 8626 (time 1787834593.66 total 110.07 last 83.30) - open/close 9401 (time 1787834603.69 total 120.10 last 77.24) total: 10000 open/close in 127.93 seconds: 78.17 ops/second llite.lustre-ffff9b94714d1000.statahead_batch_max=0 llite.lustre-ffff9b94714d1000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b94714d1000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.778s user 0m0.311s sys 0m13.458s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9b94714d1000.statahead_batch_max=256 llite.lustre-ffff9b94714d1000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b94714d1000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9b94714d1000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m48.588s user 0m0.282s sys 0m13.929s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9b94714d1000.batch_stats= snapshot_time: 1787834739.895711694 secs.nsecs start_time: 1787834679.382758516 secs.nsecs elapsed_time: 60.512953178 secs.nsecs subreqs per batch batches %% cum %% 1: 70 36 36 2: 24 12 48 4: 25 12 61 8: 8 4 65 16: 10 5 70 32: 15 7 78 64: 4 2 80 128: 2 1 81 256: 35 18 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9b94714d1000.statahead_batch_max=64 llite.lustre-ffff9b94714d1000.statahead_max=128