== sanity test 123ad: Verify batching statahead works correctly ========================================================== 10:47:55 (1787064475) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff89e6877b9800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff89e6877b9800.batch_stats=0 llite.lustre-ffff89e6877b9800.statahead_max=32 llite.lustre-ffff89e6877b9800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff89e6877b9800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff89e6877b9800.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.62 seconds: 162.12 ops/second llite.lustre-ffff89e6877b9800.statahead_max=0 101 real 0m0.589s user 0m0.002s sys 0m0.210s 'ls -l' 100 files without statahead: 0 sec llite.lustre-ffff89e6877b9800.statahead_max=32 llite.lustre-ffff89e6877b9800.statahead_max=32 101 real 0m0.403s user 0m0.004s sys 0m0.187s 'ls -l' 100 files with statahead: 0 sec statahead total: 58 statahead wrong: 2 agl total: 52 list_total: 27 fname_total: 30 hit_total: 13889 miss_total: 448 swrong=2 ewrong=2 total: 900 open/close in 1.95 seconds: 461.13 ops/second llite.lustre-ffff89e6877b9800.statahead_max=0 1001 real 0m5.707s user 0m0.017s sys 0m1.988s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff89e6877b9800.statahead_max=32 llite.lustre-ffff89e6877b9800.statahead_max=32 1001 real 0m4.047s user 0m0.009s sys 0m1.858s 'ls -l' 1000 files with statahead: 4 sec statahead total: 59 statahead wrong: 2 agl total: 53 list_total: 28 fname_total: 30 hit_total: 14888 miss_total: 449 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 6 seconds llite.lustre-ffff89e6877b9800.statahead_stats= statahead total: 59 statahead wrong: 2 agl total: 53 list_total: 28 fname_total: 30 hit_total: 14888 miss_total: 449 mdc.lustre-MDT0000-mdc-ffff89e6877b9800.batch_stats= snapshot_time: 1787064501.501770384 secs.nsecs start_time: 1787064476.547069578 secs.nsecs elapsed_time: 24.954700806 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 2 5 8 4: 3 8 16 8: 1 2 18 16: 23 62 81 32: 7 18 100 mdc.lustre-MDT0001-mdc-ffff89e6877b9800.batch_stats= snapshot_time: 1787064501.501811179 secs.nsecs start_time: 1787064476.547122283 secs.nsecs elapsed_time: 24.954688896 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 1 2 5 4: 0 0 5 8: 3 7 12 16: 13 32 45 32: 22 55 100 - open/close 4434 (time 1787064512.22 total 10.00 last 443.34) - open/close 8842 (time 1787064522.22 total 20.00 last 440.72) total: 10000 open/close in 22.67 seconds: 441.11 ops/second llite.lustre-ffff89e6877b9800.statahead_batch_max=0 llite.lustre-ffff89e6877b9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff89e6877b9800.stats=clear mdc.lustre-MDT0001-mdc-ffff89e6877b9800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m46.927s user 0m0.150s sys 0m43.047s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff89e6877b9800.statahead_batch_max=32 llite.lustre-ffff89e6877b9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff89e6877b9800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff89e6877b9800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff89e6877b9800.stats=clear mdc.lustre-MDT0001-mdc-ffff89e6877b9800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m44.946s user 0m0.148s sys 0m23.913s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff89e6877b9800.batch_stats= snapshot_time: 1787064626.089421549 secs.nsecs start_time: 1787064574.660665811 secs.nsecs elapsed_time: 51.428755738 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff89e6877b9800.batch_stats= snapshot_time: 1787064626.089484272 secs.nsecs start_time: 1787064574.660725464 secs.nsecs elapsed_time: 51.428758808 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff89e6877b9800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff89e6877b9800.batch_stats=0 llite.lustre-ffff89e6877b9800.statahead_max=2048 llite.lustre-ffff89e6877b9800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff89e6877b9800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff89e6877b9800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.14 seconds: 88.00 ops/second llite.lustre-ffff89e6877b9800.statahead_max=0 101 real 0m0.916s user 0m0.002s sys 0m0.308s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff89e6877b9800.statahead_max=2048 llite.lustre-ffff89e6877b9800.statahead_max=2048 101 real 0m0.678s user 0m0.006s sys 0m0.435s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 swrong=0 ewrong=0 total: 900 open/close in 4.43 seconds: 203.16 ops/second llite.lustre-ffff89e6877b9800.statahead_max=0 1001 real 0m10.902s user 0m0.029s sys 0m3.587s 'ls -l' 1000 files without statahead: 10 sec llite.lustre-ffff89e6877b9800.statahead_max=2048 llite.lustre-ffff89e6877b9800.statahead_max=2048 1001 real 0m4.365s user 0m0.010s sys 0m2.126s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 10 seconds llite.lustre-ffff89e6877b9800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 mdc.lustre-MDT0000-mdc-ffff89e6877b9800.batch_stats= snapshot_time: 1787064809.540085715 secs.nsecs start_time: 1787064627.321780913 secs.nsecs elapsed_time: 182.218304802 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 14 14 4: 3 42 57 8: 1 14 71 16: 0 0 71 32: 0 0 71 64: 0 0 71 128: 0 0 71 256: 2 28 100 mdc.lustre-MDT0001-mdc-ffff89e6877b9800.batch_stats= snapshot_time: 1787064809.540171215 secs.nsecs start_time: 1787064627.321906605 secs.nsecs elapsed_time: 182.218264610 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 33 33 16: 2 22 55 32: 1 11 66 64: 0 0 66 128: 1 11 77 256: 2 22 100 - open/close 2550 (time 1787064821.05 total 10.00 last 254.96) - open/close 5395 (time 1787064831.05 total 20.00 last 284.44) - open/close 7894 (time 1787064841.05 total 30.01 last 249.87) total: 10000 open/close in 38.84 seconds: 257.50 ops/second llite.lustre-ffff89e6877b9800.statahead_batch_max=0 llite.lustre-ffff89e6877b9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff89e6877b9800.stats=clear mdc.lustre-MDT0001-mdc-ffff89e6877b9800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m40.650s user 0m0.184s sys 0m34.042s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff89e6877b9800.statahead_batch_max=256 llite.lustre-ffff89e6877b9800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff89e6877b9800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff89e6877b9800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff89e6877b9800.stats=clear mdc.lustre-MDT0001-mdc-ffff89e6877b9800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.795s user 0m0.184s sys 0m21.434s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff89e6877b9800.batch_stats= snapshot_time: 1787064932.671256897 secs.nsecs start_time: 1787064894.526144416 secs.nsecs elapsed_time: 38.145112481 secs.nsecs subreqs per batch batches %% cum %% 1: 522 72 72 2: 97 13 85 4: 35 4 90 8: 6 0 91 16: 7 0 92 32: 21 2 95 64: 0 0 95 128: 1 0 95 256: 33 4 100 mdc.lustre-MDT0001-mdc-ffff89e6877b9800.batch_stats= snapshot_time: 1787064932.671306221 secs.nsecs start_time: 1787064894.526218042 secs.nsecs elapsed_time: 38.145088179 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff89e6877b9800.statahead_batch_max=64 llite.lustre-ffff89e6877b9800.statahead_max=128