== sanity test 123ad: Verify batching statahead works correctly ========================================================== 10:22:40 (1787754160) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff995ca75a8000.batch_stats=0 llite.lustre-ffff995ca75a8000.statahead_max=32 llite.lustre-ffff995ca75a8000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff995ca75a8000.batch_stats=0 total: 100 open/close in 3.61 seconds: 27.70 ops/second llite.lustre-ffff995ca75a8000.statahead_max=0 101 real 0m3.483s user 0m0.003s sys 0m0.343s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff995ca75a8000.statahead_max=32 llite.lustre-ffff995ca75a8000.statahead_max=32 101 real 0m1.811s user 0m0.008s sys 0m0.170s 'ls -l' 100 files with statahead: 1 sec statahead total: 128 statahead wrong: 2 agl total: 122 list_total: 30 fname_total: 96 hit_total: 4165 miss_total: 1285 swrong=2 ewrong=2 - open/close 479 (time 1787754195.16 total 10.01 last 47.84) total: 900 open/close in 16.63 seconds: 54.12 ops/second llite.lustre-ffff995ca75a8000.statahead_max=0 1001 real 0m23.460s user 0m0.034s sys 0m2.916s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff995ca75a8000.statahead_max=32 llite.lustre-ffff995ca75a8000.statahead_max=32 1001 real 0m11.968s user 0m0.027s sys 0m1.399s 'ls -l' 1000 files with statahead: 12 sec statahead total: 129 statahead wrong: 2 agl total: 123 list_total: 31 fname_total: 96 hit_total: 5163 miss_total: 1287 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff995ca75a8000.statahead_stats= statahead total: 129 statahead wrong: 2 agl total: 123 list_total: 31 fname_total: 96 hit_total: 5163 miss_total: 1287 mdc.lustre-MDT0000-mdc-ffff995ca75a8000.batch_stats= snapshot_time: 1787754274.177610448 secs.nsecs start_time: 1787754166.634994513 secs.nsecs elapsed_time: 107.542615935 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 415 (time 1787754288.40 total 10.03 last 41.39) - open/close 791 (time 1787754298.41 total 20.04 last 37.55) - open/close 1245 (time 1787754308.41 total 30.04 last 45.39) - open/close 1701 (time 1787754318.42 total 40.05 last 45.56) - open/close 2416 (time 1787754328.43 total 50.06 last 71.47) - open/close 3046 (time 1787754338.44 total 60.07 last 62.92) - open/close 3559 (time 1787754348.46 total 70.08 last 51.22) - open/close 4168 (time 1787754358.46 total 80.09 last 60.86) - open/close 4725 (time 1787754368.48 total 90.10 last 55.62) - open/close 5405 (time 1787754378.49 total 100.12 last 67.92) - open/close 5925 (time 1787754388.49 total 110.12 last 51.98) - open/close 6397 (time 1787754398.50 total 120.13 last 47.17) - open/close 6991 (time 1787754408.54 total 130.17 last 59.14) - open/close 7533 (time 1787754418.55 total 140.18 last 54.17) - open/close 8121 (time 1787754428.55 total 150.18 last 58.79) - open/close 8771 (time 1787754438.55 total 160.18 last 64.97) - open/close 9475 (time 1787754448.56 total 170.19 last 70.34) total: 10000 open/close in 179.49 seconds: 55.71 ops/second llite.lustre-ffff995ca75a8000.statahead_batch_max=0 llite.lustre-ffff995ca75a8000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff995ca75a8000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m12.204s user 0m0.314s sys 0m13.600s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff995ca75a8000.statahead_batch_max=32 llite.lustre-ffff995ca75a8000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff995ca75a8000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff995ca75a8000.stats=clear ls -l files in d123ad.sanity: 10001 real 2m8.466s user 0m0.349s sys 0m13.445s batched statahead RPC= hit count=9979 mdc.lustre-MDT0000-mdc-ffff995ca75a8000.batch_stats= snapshot_time: 1787754681.976715690 secs.nsecs start_time: 1787754540.408671162 secs.nsecs elapsed_time: 141.568044528 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff995ca75a8000.batch_stats=0 llite.lustre-ffff995ca75a8000.statahead_max=2048 llite.lustre-ffff995ca75a8000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff995ca75a8000.batch_stats=0 total: 100 open/close in 3.76 seconds: 26.59 ops/second llite.lustre-ffff995ca75a8000.statahead_max=0 101 real 0m2.643s user 0m0.003s sys 0m0.311s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff995ca75a8000.statahead_max=2048 llite.lustre-ffff995ca75a8000.statahead_max=2048 101 real 0m1.031s user 0m0.006s sys 0m0.155s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10078 miss_total: 22 swrong=0 ewrong=0 - open/close 505 (time 1787755017.36 total 10.03 last 50.35) total: 900 open/close in 16.85 seconds: 53.40 ops/second llite.lustre-ffff995ca75a8000.statahead_max=0 1001 real 0m31.564s user 0m0.040s sys 0m2.980s 'ls -l' 1000 files without statahead: 31 sec llite.lustre-ffff995ca75a8000.statahead_max=2048 llite.lustre-ffff995ca75a8000.statahead_max=2048 1001 real 0m10.790s user 0m0.023s sys 0m1.321s 'ls -l' 1000 files with statahead: 10 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff995ca75a8000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 mdc.lustre-MDT0000-mdc-ffff995ca75a8000.batch_stats= snapshot_time: 1787755112.665827230 secs.nsecs start_time: 1787754686.101858529 secs.nsecs elapsed_time: 426.563968701 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 683 (time 1787755125.46 total 10.00 last 68.28) - open/close 1279 (time 1787755135.47 total 20.01 last 59.56) - open/close 1830 (time 1787755145.47 total 30.02 last 55.07) - open/close 2426 (time 1787755155.48 total 40.02 last 59.57) - open/close 3031 (time 1787755165.51 total 50.05 last 60.34) - open/close 3729 (time 1787755175.51 total 60.05 last 69.78) - open/close 4468 (time 1787755185.53 total 70.07 last 73.78) - open/close 5271 (time 1787755195.53 total 80.07 last 80.30) - open/close 5906 (time 1787755205.55 total 90.09 last 63.37) - open/close 6425 (time 1787755215.55 total 100.09 last 51.88) - open/close 6965 (time 1787755225.56 total 110.10 last 53.97) - open/close 7553 (time 1787755235.56 total 120.10 last 58.76) - open/close 8175 (time 1787755245.58 total 130.12 last 62.11) - open/close 8745 (time 1787755255.60 total 140.14 last 56.88) - open/close 9375 (time 1787755265.61 total 150.15 last 62.94) - open/close 9828 (time 1787755275.61 total 160.16 last 45.27) total: 10000 open/close in 162.02 seconds: 61.72 ops/second llite.lustre-ffff995ca75a8000.statahead_batch_max=0 llite.lustre-ffff995ca75a8000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff995ca75a8000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.697s user 0m0.306s sys 0m13.617s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff995ca75a8000.statahead_batch_max=256 llite.lustre-ffff995ca75a8000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff995ca75a8000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff995ca75a8000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.691s user 0m0.283s sys 0m12.641s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff995ca75a8000.batch_stats= snapshot_time: 1787755426.909760262 secs.nsecs start_time: 1787755353.644595381 secs.nsecs elapsed_time: 73.265164881 secs.nsecs subreqs per batch batches %% cum %% 1: 54 34 34 2: 19 12 46 4: 13 8 55 8: 5 3 58 16: 6 3 62 32: 20 12 75 64: 3 1 76 128: 0 0 76 256: 36 23 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff995ca75a8000.statahead_batch_max=64 llite.lustre-ffff995ca75a8000.statahead_max=128