== sanity test 123ad: Verify batching statahead works correctly ========================================================== 22:15:03 (1787796903) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff92040710e000.batch_stats=0 llite.lustre-ffff92040710e000.statahead_max=32 llite.lustre-ffff92040710e000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff92040710e000.batch_stats=0 total: 100 open/close in 2.39 seconds: 41.77 ops/second llite.lustre-ffff92040710e000.statahead_max=0 101 real 0m2.646s user 0m0.010s sys 0m0.308s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff92040710e000.statahead_max=32 llite.lustre-ffff92040710e000.statahead_max=32 101 real 0m1.135s user 0m0.006s sys 0m0.142s 'ls -l' 100 files with statahead: 1 sec statahead total: 130 statahead wrong: 2 agl total: 125 list_total: 29 fname_total: 86 hit_total: 3909 miss_total: 1423 swrong=2 ewrong=2 - open/close 689 (time 1787796931.26 total 10.03 last 68.70) total: 900 open/close in 13.61 seconds: 66.11 ops/second llite.lustre-ffff92040710e000.statahead_max=0 1001 real 0m21.325s user 0m0.033s sys 0m2.740s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff92040710e000.statahead_max=32 llite.lustre-ffff92040710e000.statahead_max=32 1001 real 0m12.570s user 0m0.027s sys 0m1.338s 'ls -l' 1000 files with statahead: 13 sec statahead total: 131 statahead wrong: 2 agl total: 126 list_total: 30 fname_total: 86 hit_total: 4906 miss_total: 1426 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff92040710e000.statahead_stats= statahead total: 131 statahead wrong: 2 agl total: 126 list_total: 30 fname_total: 86 hit_total: 4906 miss_total: 1426 mdc.lustre-MDT0000-mdc-ffff92040710e000.batch_stats= snapshot_time: 1787797008.975871521 secs.nsecs start_time: 1787796908.830804932 secs.nsecs elapsed_time: 100.145066589 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 713 (time 1787797022.03 total 10.01 last 71.24) - open/close 1388 (time 1787797032.03 total 20.01 last 67.46) - open/close 2011 (time 1787797042.04 total 30.02 last 62.27) - open/close 2520 (time 1787797052.05 total 40.03 last 50.85) - open/close 3246 (time 1787797062.05 total 50.03 last 72.59) - open/close 3921 (time 1787797072.05 total 60.03 last 67.47) - open/close 4614 (time 1787797082.06 total 70.04 last 69.25) - open/close 5358 (time 1787797092.06 total 80.04 last 74.38) - open/close 6115 (time 1787797102.06 total 90.05 last 75.69) - open/close 6798 (time 1787797112.07 total 100.06 last 68.23) - open/close 7619 (time 1787797122.08 total 110.06 last 82.07) - open/close 8448 (time 1787797132.08 total 120.06 last 82.89) - open/close 9209 (time 1787797142.08 total 130.06 last 76.09) - open/close 9918 (time 1787797152.09 total 140.07 last 70.86) total: 10000 open/close in 141.04 seconds: 70.90 ops/second llite.lustre-ffff92040710e000.statahead_batch_max=0 llite.lustre-ffff92040710e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92040710e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.213s user 0m0.299s sys 0m13.452s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92040710e000.statahead_batch_max=32 llite.lustre-ffff92040710e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92040710e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92040710e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m45.181s user 0m0.284s sys 0m12.672s batched statahead RPC= hit count=9982 mdc.lustre-MDT0000-mdc-ffff92040710e000.batch_stats= snapshot_time: 1787797346.077283374 secs.nsecs start_time: 1787797228.231072895 secs.nsecs elapsed_time: 117.846210479 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff92040710e000.batch_stats=0 llite.lustre-ffff92040710e000.statahead_max=2048 llite.lustre-ffff92040710e000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff92040710e000.batch_stats=0 total: 100 open/close in 2.99 seconds: 33.43 ops/second llite.lustre-ffff92040710e000.statahead_max=0 101 real 0m2.752s user 0m0.003s sys 0m0.316s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff92040710e000.statahead_max=2048 llite.lustre-ffff92040710e000.statahead_max=2048 101 real 0m1.290s user 0m0.004s sys 0m0.173s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10081 miss_total: 19 swrong=0 ewrong=0 - open/close 700 (time 1787797666.79 total 10.00 last 69.98) total: 900 open/close in 13.10 seconds: 68.68 ops/second llite.lustre-ffff92040710e000.statahead_max=0 1001 real 0m20.684s user 0m0.035s sys 0m2.835s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff92040710e000.statahead_max=2048 llite.lustre-ffff92040710e000.statahead_max=2048 1001 real 0m6.232s user 0m0.026s sys 0m1.238s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff92040710e000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 mdc.lustre-MDT0000-mdc-ffff92040710e000.batch_stats= snapshot_time: 1787797731.117610278 secs.nsecs start_time: 1787797349.256511742 secs.nsecs elapsed_time: 381.861098536 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 3 25 58 64: 1 8 66 128: 0 0 66 256: 4 33 100 - open/close 727 (time 1787797743.63 total 10.00 last 72.67) - open/close 1521 (time 1787797753.63 total 20.00 last 79.40) - open/close 2259 (time 1787797763.64 total 30.02 last 73.71) - open/close 2918 (time 1787797773.66 total 40.04 last 65.77) - open/close 3453 (time 1787797783.67 total 50.04 last 53.47) - open/close 4160 (time 1787797793.67 total 60.04 last 70.69) - open/close 5020 (time 1787797803.69 total 70.06 last 85.83) - open/close 5703 (time 1787797813.69 total 80.07 last 68.29) - open/close 6457 (time 1787797823.70 total 90.07 last 75.34) - open/close 7386 (time 1787797833.70 total 100.08 last 92.87) - open/close 8159 (time 1787797843.71 total 110.08 last 77.25) - open/close 8951 (time 1787797853.71 total 120.09 last 79.17) - open/close 9596 (time 1787797863.72 total 130.10 last 64.45) total: 10000 open/close in 136.67 seconds: 73.17 ops/second llite.lustre-ffff92040710e000.statahead_batch_max=0 llite.lustre-ffff92040710e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92040710e000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.285s user 0m0.292s sys 0m13.610s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92040710e000.statahead_batch_max=256 llite.lustre-ffff92040710e000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92040710e000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92040710e000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.173s user 0m0.282s sys 0m13.803s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff92040710e000.batch_stats= snapshot_time: 1787798000.331516063 secs.nsecs start_time: 1787797939.469540795 secs.nsecs elapsed_time: 60.861975268 secs.nsecs subreqs per batch batches %% cum %% 1: 89 39 39 2: 36 15 55 4: 21 9 64 8: 15 6 70 16: 9 3 74 32: 17 7 82 64: 4 1 84 128: 1 0 84 256: 35 15 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff92040710e000.statahead_batch_max=64 llite.lustre-ffff92040710e000.statahead_max=128