== sanity test 123ad: Verify batching statahead works correctly ========================================================== 20:07:28 (1787702848) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff930c44c8f800.batch_stats=0 llite.lustre-ffff930c44c8f800.statahead_max=32 llite.lustre-ffff930c44c8f800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff930c44c8f800.batch_stats=0 total: 100 open/close in 3.31 seconds: 30.18 ops/second llite.lustre-ffff930c44c8f800.statahead_max=0 101 real 0m2.677s user 0m0.008s sys 0m0.356s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff930c44c8f800.statahead_max=32 llite.lustre-ffff930c44c8f800.statahead_max=32 101 real 0m1.477s user 0m0.006s sys 0m0.174s 'ls -l' 100 files with statahead: 2 sec statahead total: 128 statahead wrong: 2 agl total: 122 list_total: 30 fname_total: 84 hit_total: 4089 miss_total: 1283 swrong=2 ewrong=2 - open/close 634 (time 1787702878.16 total 10.01 last 63.35) total: 900 open/close in 14.80 seconds: 60.83 ops/second llite.lustre-ffff930c44c8f800.statahead_max=0 1001 real 0m23.203s user 0m0.028s sys 0m2.752s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff930c44c8f800.statahead_max=32 llite.lustre-ffff930c44c8f800.statahead_max=32 1001 real 0m12.124s user 0m0.027s sys 0m1.279s 'ls -l' 1000 files with statahead: 12 sec statahead total: 129 statahead wrong: 2 agl total: 123 list_total: 31 fname_total: 84 hit_total: 5088 miss_total: 1284 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff930c44c8f800.statahead_stats= statahead total: 129 statahead wrong: 2 agl total: 123 list_total: 31 fname_total: 84 hit_total: 5088 miss_total: 1284 mdc.lustre-MDT0000-mdc-ffff930c44c8f800.batch_stats= snapshot_time: 1787702960.233108439 secs.nsecs start_time: 1787702853.627656343 secs.nsecs elapsed_time: 106.605452096 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 1 2 5 8: 2 5 10 16: 2 5 15 32: 34 85 100 - open/close 628 (time 1787702973.16 total 10.01 last 62.74) - open/close 1292 (time 1787702983.17 total 20.02 last 66.31) - open/close 1952 (time 1787702993.17 total 30.02 last 65.99) - open/close 2590 (time 1787703003.19 total 40.05 last 63.66) - open/close 3243 (time 1787703013.21 total 50.06 last 65.19) - open/close 3846 (time 1787703023.22 total 60.07 last 60.24) - open/close 4444 (time 1787703033.23 total 70.08 last 59.74) - open/close 4985 (time 1787703043.24 total 80.09 last 54.05) - open/close 5502 (time 1787703053.25 total 90.10 last 51.67) - open/close 6036 (time 1787703063.28 total 100.13 last 53.25) - open/close 6585 (time 1787703073.29 total 110.14 last 54.85) - open/close 7084 (time 1787703083.30 total 120.16 last 49.81) - open/close 7661 (time 1787703093.31 total 130.17 last 57.64) - open/close 8278 (time 1787703103.32 total 140.17 last 61.67) - open/close 8976 (time 1787703113.33 total 150.18 last 69.71) - open/close 9538 (time 1787703123.35 total 160.20 last 56.12) total: 10000 open/close in 167.48 seconds: 59.71 ops/second llite.lustre-ffff930c44c8f800.statahead_batch_max=0 llite.lustre-ffff930c44c8f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff930c44c8f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.412s user 0m0.257s sys 0m13.141s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff930c44c8f800.statahead_batch_max=32 llite.lustre-ffff930c44c8f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff930c44c8f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff930c44c8f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m36.685s user 0m0.250s sys 0m12.922s batched statahead RPC= hit count=9985 mdc.lustre-MDT0000-mdc-ffff930c44c8f800.batch_stats= snapshot_time: 1787703314.585817879 secs.nsecs start_time: 1787703207.360867311 secs.nsecs elapsed_time: 107.224950568 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff930c44c8f800.batch_stats=0 llite.lustre-ffff930c44c8f800.statahead_max=2048 llite.lustre-ffff930c44c8f800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff930c44c8f800.batch_stats=0 total: 100 open/close in 2.96 seconds: 33.76 ops/second llite.lustre-ffff930c44c8f800.statahead_max=0 101 real 0m1.701s user 0m0.005s sys 0m0.291s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff930c44c8f800.statahead_max=2048 llite.lustre-ffff930c44c8f800.statahead_max=2048 101 real 0m1.036s user 0m0.003s sys 0m0.174s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10084 miss_total: 16 swrong=0 ewrong=0 - open/close 746 (time 1787703662.52 total 10.01 last 74.56) total: 900 open/close in 11.89 seconds: 75.67 ops/second llite.lustre-ffff930c44c8f800.statahead_max=0 1001 real 0m21.193s user 0m0.038s sys 0m2.985s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff930c44c8f800.statahead_max=2048 llite.lustre-ffff930c44c8f800.statahead_max=2048 1001 real 0m6.532s user 0m0.024s sys 0m1.221s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff930c44c8f800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 mdc.lustre-MDT0000-mdc-ffff930c44c8f800.batch_stats= snapshot_time: 1787703731.790411647 secs.nsecs start_time: 1787703317.276788169 secs.nsecs elapsed_time: 414.513623478 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 633 (time 1787703744.80 total 10.00 last 63.29) - open/close 1237 (time 1787703754.81 total 20.01 last 60.34) - open/close 1922 (time 1787703764.82 total 30.02 last 68.45) - open/close 2677 (time 1787703774.82 total 40.03 last 75.45) - open/close 3345 (time 1787703784.83 total 50.03 last 66.76) - open/close 3972 (time 1787703794.83 total 60.04 last 62.66) - open/close 4650 (time 1787703804.86 total 70.06 last 67.65) - open/close 5254 (time 1787703814.86 total 80.06 last 60.39) - open/close 5842 (time 1787703824.86 total 90.07 last 58.77) - open/close 6470 (time 1787703834.89 total 100.10 last 62.62) - open/close 6921 (time 1787703844.91 total 110.11 last 45.03) - open/close 7547 (time 1787703854.92 total 120.12 last 62.53) - open/close 8178 (time 1787703864.93 total 130.13 last 63.04) - open/close 8840 (time 1787703874.94 total 140.15 last 66.09) - open/close 9408 (time 1787703884.97 total 150.17 last 56.65) - open/close 9974 (time 1787703894.98 total 160.18 last 56.56) total: 10000 open/close in 160.60 seconds: 62.27 ops/second llite.lustre-ffff930c44c8f800.statahead_batch_max=0 llite.lustre-ffff930c44c8f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff930c44c8f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.044s user 0m0.298s sys 0m13.500s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff930c44c8f800.statahead_batch_max=256 llite.lustre-ffff930c44c8f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff930c44c8f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff930c44c8f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.486s user 0m0.288s sys 0m15.002s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff930c44c8f800.batch_stats= snapshot_time: 1787704036.427138885 secs.nsecs start_time: 1787703975.774394090 secs.nsecs elapsed_time: 60.652744795 secs.nsecs subreqs per batch batches %% cum %% 1: 75 44 44 2: 15 8 53 4: 10 5 59 8: 4 2 61 16: 10 5 67 32: 16 9 76 64: 3 1 78 128: 0 0 78 256: 36 21 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff930c44c8f800.statahead_batch_max=64 llite.lustre-ffff930c44c8f800.statahead_max=128