== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:29:09 (1787578149) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.batch_stats=0 llite.lustre-ffff93e47e4e1800.statahead_max=32 llite.lustre-ffff93e47e4e1800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.batch_stats=0 total: 100 open/close in 3.04 seconds: 32.85 ops/second llite.lustre-ffff93e47e4e1800.statahead_max=0 101 real 0m2.813s user 0m0.009s sys 0m0.346s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff93e47e4e1800.statahead_max=32 llite.lustre-ffff93e47e4e1800.statahead_max=32 101 real 0m0.847s user 0m0.006s sys 0m0.179s 'ls -l' 100 files with statahead: 1 sec statahead total: 137 statahead wrong: 2 agl total: 131 list_total: 30 fname_total: 107 hit_total: 3996 miss_total: 1433 swrong=2 ewrong=2 - open/close 734 (time 1787578179.91 total 10.00 last 73.37) total: 900 open/close in 13.21 seconds: 68.14 ops/second llite.lustre-ffff93e47e4e1800.statahead_max=0 1001 real 0m34.307s user 0m0.049s sys 0m3.083s 'ls -l' 1000 files without statahead: 34 sec llite.lustre-ffff93e47e4e1800.statahead_max=32 llite.lustre-ffff93e47e4e1800.statahead_max=32 1001 real 0m12.208s user 0m0.031s sys 0m1.489s 'ls -l' 1000 files with statahead: 13 sec statahead total: 138 statahead wrong: 2 agl total: 132 list_total: 31 fname_total: 107 hit_total: 4990 miss_total: 1439 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff93e47e4e1800.statahead_stats= statahead total: 138 statahead wrong: 2 agl total: 132 list_total: 31 fname_total: 107 hit_total: 4990 miss_total: 1439 mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.batch_stats= snapshot_time: 1787578267.881071253 secs.nsecs start_time: 1787578154.487427679 secs.nsecs elapsed_time: 113.393643574 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 2 5 7 16: 3 7 15 32: 34 85 100 - open/close 445 (time 1787578282.25 total 10.04 last 44.32) - open/close 813 (time 1787578292.26 total 20.05 last 36.76) - open/close 1458 (time 1787578302.27 total 30.07 last 64.41) - open/close 2047 (time 1787578312.27 total 40.07 last 58.89) - open/close 2652 (time 1787578322.29 total 50.08 last 60.44) - open/close 3233 (time 1787578332.29 total 60.09 last 58.05) - open/close 3898 (time 1787578342.31 total 70.10 last 66.41) - open/close 4456 (time 1787578352.32 total 80.12 last 55.71) - open/close 5027 (time 1787578362.33 total 90.12 last 57.09) - open/close 5681 (time 1787578372.34 total 100.13 last 65.31) - open/close 6298 (time 1787578382.34 total 110.14 last 61.67) - open/close 6995 (time 1787578392.35 total 120.14 last 69.65) - open/close 7557 (time 1787578402.35 total 130.15 last 56.18) - open/close 8097 (time 1787578412.39 total 140.18 last 53.81) - open/close 8727 (time 1787578422.39 total 150.19 last 62.97) - open/close 9351 (time 1787578432.41 total 160.20 last 62.31) - open/close 9964 (time 1787578442.42 total 170.21 last 61.25) total: 10000 open/close in 170.78 seconds: 58.55 ops/second llite.lustre-ffff93e47e4e1800.statahead_batch_max=0 llite.lustre-ffff93e47e4e1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.891s user 0m0.330s sys 0m16.077s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93e47e4e1800.statahead_batch_max=32 llite.lustre-ffff93e47e4e1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m56.090s user 0m0.297s sys 0m13.901s batched statahead RPC= hit count=9977 mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.batch_stats= snapshot_time: 1787578650.536470629 secs.nsecs start_time: 1787578524.806938394 secs.nsecs elapsed_time: 125.729532235 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 0 0 4: 0 0 0 8: 1 0 0 16: 0 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.batch_stats=0 llite.lustre-ffff93e47e4e1800.statahead_max=2048 llite.lustre-ffff93e47e4e1800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.batch_stats=0 total: 100 open/close in 3.09 seconds: 32.32 ops/second llite.lustre-ffff93e47e4e1800.statahead_max=0 101 real 0m2.018s user 0m0.005s sys 0m0.294s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff93e47e4e1800.statahead_max=2048 llite.lustre-ffff93e47e4e1800.statahead_max=2048 101 real 0m0.925s user 0m0.005s sys 0m0.190s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10076 miss_total: 24 swrong=0 ewrong=0 - open/close 694 (time 1787579028.18 total 10.00 last 69.37) total: 900 open/close in 13.31 seconds: 67.64 ops/second llite.lustre-ffff93e47e4e1800.statahead_max=0 1001 real 0m28.126s user 0m0.035s sys 0m2.973s 'ls -l' 1000 files without statahead: 28 sec llite.lustre-ffff93e47e4e1800.statahead_max=2048 llite.lustre-ffff93e47e4e1800.statahead_max=2048 1001 real 0m9.924s user 0m0.030s sys 0m1.569s 'ls -l' 1000 files with statahead: 10 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11075 miss_total: 25 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff93e47e4e1800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11075 miss_total: 25 mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.batch_stats= snapshot_time: 1787579111.620137304 secs.nsecs start_time: 1787578653.871703279 secs.nsecs elapsed_time: 457.748434025 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 2 22 44 32: 0 0 44 64: 0 0 44 128: 1 11 55 256: 4 44 100 - open/close 374 (time 1787579126.48 total 10.02 last 37.31) - open/close 792 (time 1787579136.49 total 20.04 last 41.75) - open/close 1362 (time 1787579146.50 total 30.05 last 56.95) - open/close 2038 (time 1787579156.51 total 40.06 last 67.52) - open/close 2556 (time 1787579166.51 total 50.06 last 51.80) - open/close 3068 (time 1787579176.52 total 60.06 last 51.19) - open/close 3699 (time 1787579186.52 total 70.07 last 63.06) - open/close 4162 (time 1787579196.55 total 80.09 last 46.18) - open/close 4705 (time 1787579206.55 total 90.10 last 54.26) - open/close 5158 (time 1787579216.55 total 100.10 last 45.30) - open/close 5686 (time 1787579226.56 total 110.11 last 52.77) - open/close 6150 (time 1787579236.61 total 120.15 last 46.18) - open/close 6782 (time 1787579246.61 total 130.16 last 63.17) - open/close 7305 (time 1787579256.63 total 140.17 last 52.23) - open/close 7931 (time 1787579266.63 total 150.17 last 62.59) - open/close 8606 (time 1787579276.63 total 160.18 last 67.48) - open/close 9131 (time 1787579286.66 total 170.21 last 52.33) - open/close 9844 (time 1787579296.67 total 180.22 last 71.26) total: 10000 open/close in 182.76 seconds: 54.72 ops/second llite.lustre-ffff93e47e4e1800.statahead_batch_max=0 llite.lustre-ffff93e47e4e1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.129s user 0m0.355s sys 0m16.067s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93e47e4e1800.statahead_batch_max=256 llite.lustre-ffff93e47e4e1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m48.178s user 0m0.279s sys 0m15.944s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff93e47e4e1800.batch_stats= snapshot_time: 1787579429.124920799 secs.nsecs start_time: 1787579370.301073944 secs.nsecs elapsed_time: 58.823846855 secs.nsecs subreqs per batch batches %% cum %% 1: 174 48 48 2: 40 11 59 4: 52 14 73 8: 20 5 79 16: 20 5 84 32: 18 4 89 64: 4 1 90 128: 0 0 90 256: 34 9 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff93e47e4e1800.statahead_batch_max=64 llite.lustre-ffff93e47e4e1800.statahead_max=128