== sanity test 123ad: Verify batching statahead works correctly ========================================================== 12:42:09 (1786207329) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff95aa20220000.batch_stats=0 llite.lustre-ffff95aa20220000.statahead_max=32 llite.lustre-ffff95aa20220000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff95aa20220000.batch_stats=0 total: 100 open/close in 2.41 seconds: 41.47 ops/second llite.lustre-ffff95aa20220000.statahead_max=0 101 real 0m2.559s user 0m0.003s sys 0m0.363s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff95aa20220000.statahead_max=32 llite.lustre-ffff95aa20220000.statahead_max=32 101 real 0m1.533s user 0m0.004s sys 0m0.200s 'ls -l' 100 files with statahead: 1 sec statahead total: 76 statahead wrong: 2 agl total: 71 list_total: 29 fname_total: 46 hit_total: 13791 miss_total: 875 swrong=2 ewrong=2 - open/close 714 (time 1786207357.69 total 10.00 last 71.37) total: 900 open/close in 14.23 seconds: 63.26 ops/second llite.lustre-ffff95aa20220000.statahead_max=0 1001 real 0m34.918s user 0m0.035s sys 0m3.259s 'ls -l' 1000 files without statahead: 35 sec llite.lustre-ffff95aa20220000.statahead_max=32 llite.lustre-ffff95aa20220000.statahead_max=32 1001 real 0m12.491s user 0m0.023s sys 0m1.448s 'ls -l' 1000 files with statahead: 12 sec statahead total: 77 statahead wrong: 2 agl total: 72 list_total: 30 fname_total: 46 hit_total: 14790 miss_total: 876 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff95aa20220000.statahead_stats= statahead total: 77 statahead wrong: 2 agl total: 72 list_total: 30 fname_total: 46 hit_total: 14790 miss_total: 876 mdc.lustre-MDT0000-mdc-ffff95aa20220000.batch_stats= snapshot_time: 1786207451.450546477 secs.nsecs start_time: 1786207334.227628450 secs.nsecs elapsed_time: 117.222918027 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 708 (time 1786207463.96 total 10.01 last 70.75) - open/close 1418 (time 1786207473.97 total 20.01 last 70.96) - open/close 2062 (time 1786207483.97 total 30.02 last 64.36) - open/close 2632 (time 1786207493.97 total 40.02 last 57.00) - open/close 3286 (time 1786207503.98 total 50.03 last 65.37) - open/close 3960 (time 1786207513.99 total 60.04 last 67.29) - open/close 4591 (time 1786207524.01 total 70.05 last 63.02) - open/close 5214 (time 1786207534.02 total 80.07 last 62.23) - open/close 5869 (time 1786207544.02 total 90.07 last 65.49) - open/close 6509 (time 1786207554.03 total 100.08 last 63.91) - open/close 7255 (time 1786207564.07 total 110.11 last 74.36) - open/close 7945 (time 1786207574.09 total 120.14 last 68.83) - open/close 8541 (time 1786207584.10 total 130.14 last 59.56) - open/close 9267 (time 1786207594.11 total 140.16 last 72.52) - open/close 9914 (time 1786207604.11 total 150.16 last 64.70) total: 10000 open/close in 151.62 seconds: 65.95 ops/second llite.lustre-ffff95aa20220000.statahead_batch_max=0 llite.lustre-ffff95aa20220000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95aa20220000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.749s user 0m0.301s sys 0m15.637s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95aa20220000.statahead_batch_max=32 llite.lustre-ffff95aa20220000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95aa20220000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95aa20220000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m40.196s user 0m0.310s sys 0m15.120s batched statahead RPC= hit count=9982 mdc.lustre-MDT0000-mdc-ffff95aa20220000.batch_stats= snapshot_time: 1786207801.668827718 secs.nsecs start_time: 1786207687.653714458 secs.nsecs elapsed_time: 114.015113260 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff95aa20220000.batch_stats=0 llite.lustre-ffff95aa20220000.statahead_max=2048 llite.lustre-ffff95aa20220000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff95aa20220000.batch_stats=0 total: 100 open/close in 3.82 seconds: 26.19 ops/second llite.lustre-ffff95aa20220000.statahead_max=0 101 real 0m4.413s user 0m0.004s sys 0m0.353s 'ls -l' 100 files without statahead: 5 sec llite.lustre-ffff95aa20220000.statahead_max=2048 llite.lustre-ffff95aa20220000.statahead_max=2048 101 real 0m1.610s user 0m0.004s sys 0m0.181s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10081 miss_total: 19 swrong=0 ewrong=0 - open/close 768 (time 1786208128.58 total 10.00 last 76.76) total: 900 open/close in 11.56 seconds: 77.85 ops/second llite.lustre-ffff95aa20220000.statahead_max=0 1001 real 0m18.294s user 0m0.032s sys 0m3.041s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff95aa20220000.statahead_max=2048 llite.lustre-ffff95aa20220000.statahead_max=2048 1001 real 0m7.321s user 0m0.028s sys 0m1.632s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 17 seconds llite.lustre-ffff95aa20220000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 mdc.lustre-MDT0000-mdc-ffff95aa20220000.batch_stats= snapshot_time: 1786208183.719571063 secs.nsecs start_time: 1786207804.095178444 secs.nsecs elapsed_time: 379.624392619 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 1 11 33 32: 1 11 44 64: 1 11 55 128: 0 0 55 256: 4 44 100 - open/close 787 (time 1786208196.01 total 10.00 last 78.69) - open/close 1495 (time 1786208206.01 total 20.00 last 70.78) - open/close 2359 (time 1786208216.01 total 30.01 last 86.36) - open/close 3215 (time 1786208226.03 total 40.02 last 85.46) - open/close 4078 (time 1786208236.04 total 50.03 last 86.25) - open/close 4737 (time 1786208246.04 total 60.03 last 65.89) - open/close 5510 (time 1786208256.04 total 70.04 last 77.26) - open/close 6251 (time 1786208266.05 total 80.04 last 74.07) - open/close 6975 (time 1786208276.05 total 90.05 last 72.34) - open/close 7894 (time 1786208286.05 total 100.05 last 91.90) - open/close 8668 (time 1786208296.07 total 110.06 last 77.29) - open/close 9418 (time 1786208306.07 total 120.07 last 74.96) total: 10000 open/close in 127.27 seconds: 78.57 ops/second llite.lustre-ffff95aa20220000.statahead_batch_max=0 llite.lustre-ffff95aa20220000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95aa20220000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.635s user 0m0.310s sys 0m15.539s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95aa20220000.statahead_batch_max=256 llite.lustre-ffff95aa20220000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95aa20220000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95aa20220000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.571s user 0m0.310s sys 0m15.664s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff95aa20220000.batch_stats= snapshot_time: 1786208447.710285667 secs.nsecs start_time: 1786208378.136961780 secs.nsecs elapsed_time: 69.573323887 secs.nsecs subreqs per batch batches %% cum %% 1: 243 50 50 2: 75 15 66 4: 53 11 77 8: 35 7 84 16: 22 4 89 32: 12 2 92 64: 4 0 92 128: 1 0 93 256: 33 6 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff95aa20220000.statahead_batch_max=64 llite.lustre-ffff95aa20220000.statahead_max=128