== sanity test 123ad: Verify batching statahead works correctly ========================================================== 02:33:58 (1784702038) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8b8d47221800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8b8d47221800.batch_stats=0 llite.lustre-ffff8b8d47221800.statahead_max=32 llite.lustre-ffff8b8d47221800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8b8d47221800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8b8d47221800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.11 seconds: 47.29 ops/second llite.lustre-ffff8b8d47221800.statahead_max=0 101 real 0m2.044s user 0m0.003s sys 0m0.338s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8b8d47221800.statahead_max=32 llite.lustre-ffff8b8d47221800.statahead_max=32 101 real 0m0.715s user 0m0.008s sys 0m0.207s 'ls -l' 100 files with statahead: 1 sec statahead total: 109 statahead wrong: 2 agl total: 105 list_total: 31 fname_total: 66 hit_total: 3801 miss_total: 1710 swrong=2 ewrong=2 total: 900 open/close in 8.76 seconds: 102.69 ops/second llite.lustre-ffff8b8d47221800.statahead_max=0 1001 real 0m16.344s user 0m0.038s sys 0m3.108s 'ls -l' 1000 files without statahead: 16 sec llite.lustre-ffff8b8d47221800.statahead_max=32 llite.lustre-ffff8b8d47221800.statahead_max=32 1001 real 0m4.962s user 0m0.022s sys 0m1.717s 'ls -l' 1000 files with statahead: 5 sec statahead total: 110 statahead wrong: 2 agl total: 106 list_total: 32 fname_total: 66 hit_total: 4798 miss_total: 1713 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 10 seconds llite.lustre-ffff8b8d47221800.statahead_stats= statahead total: 110 statahead wrong: 2 agl total: 106 list_total: 32 fname_total: 66 hit_total: 4798 miss_total: 1713 mdc.lustre-MDT0000-mdc-ffff8b8d47221800.batch_stats= snapshot_time: 1784702096.964835863 secs.nsecs start_time: 1784702041.593905926 secs.nsecs elapsed_time: 55.370929937 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 5 5 8: 4 10 15 16: 20 51 66 32: 13 33 100 mdc.lustre-MDT0001-mdc-ffff8b8d47221800.batch_stats= snapshot_time: 1784702096.964903065 secs.nsecs start_time: 1784702041.594179838 secs.nsecs elapsed_time: 55.370723227 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 1 2 5 4: 2 5 10 8: 1 2 12 16: 23 58 71 32: 11 28 100 - open/close 1214 (time 1784702108.31 total 10.00 last 121.39) - open/close 2428 (time 1784702118.32 total 20.00 last 121.38) - open/close 3650 (time 1784702128.32 total 30.00 last 122.18) - open/close 4778 (time 1784702138.32 total 40.01 last 112.71) - open/close 5886 (time 1784702148.33 total 50.01 last 110.76) - open/close 7070 (time 1784702158.33 total 60.02 last 118.37) - open/close 8343 (time 1784702168.33 total 70.02 last 127.27) - open/close 9625 (time 1784702178.33 total 80.02 last 128.18) total: 10000 open/close in 83.24 seconds: 120.14 ops/second llite.lustre-ffff8b8d47221800.statahead_batch_max=0 llite.lustre-ffff8b8d47221800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b8d47221800.stats=clear mdc.lustre-MDT0001-mdc-ffff8b8d47221800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m37.833s user 0m0.269s sys 0m26.468s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b8d47221800.statahead_batch_max=32 llite.lustre-ffff8b8d47221800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b8d47221800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8b8d47221800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b8d47221800.stats=clear mdc.lustre-MDT0001-mdc-ffff8b8d47221800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.235s user 0m0.243s sys 0m18.402s batched statahead RPC= hit count=9982 mdc.lustre-MDT0000-mdc-ffff8b8d47221800.batch_stats= snapshot_time: 1784702287.469278638 secs.nsecs start_time: 1784702224.470990482 secs.nsecs elapsed_time: 62.998288156 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff8b8d47221800.batch_stats= snapshot_time: 1784702287.469342755 secs.nsecs start_time: 1784702224.471057792 secs.nsecs elapsed_time: 62.998284963 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8b8d47221800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8b8d47221800.batch_stats=0 llite.lustre-ffff8b8d47221800.statahead_max=2048 llite.lustre-ffff8b8d47221800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8b8d47221800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8b8d47221800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.35 seconds: 74.17 ops/second llite.lustre-ffff8b8d47221800.statahead_max=0 101 real 0m1.372s user 0m0.005s sys 0m0.323s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff8b8d47221800.statahead_max=2048 llite.lustre-ffff8b8d47221800.statahead_max=2048 101 real 0m0.536s user 0m0.002s sys 0m0.244s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10081 miss_total: 19 swrong=0 ewrong=0 total: 900 open/close in 6.06 seconds: 148.56 ops/second llite.lustre-ffff8b8d47221800.statahead_max=0 1001 real 0m12.671s user 0m0.023s sys 0m2.932s 'ls -l' 1000 files without statahead: 12 sec llite.lustre-ffff8b8d47221800.statahead_max=2048 llite.lustre-ffff8b8d47221800.statahead_max=2048 1001 real 0m3.765s user 0m0.022s sys 0m1.694s 'ls -l' 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 10 seconds llite.lustre-ffff8b8d47221800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 mdc.lustre-MDT0000-mdc-ffff8b8d47221800.batch_stats= snapshot_time: 1784702497.990586591 secs.nsecs start_time: 1784702288.706555841 secs.nsecs elapsed_time: 209.284030750 secs.nsecs subreqs per batch batches %% cum %% 1: 1 12 12 2: 2 25 37 4: 1 12 50 8: 2 25 75 16: 0 0 75 32: 0 0 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff8b8d47221800.batch_stats= snapshot_time: 1784702497.990649215 secs.nsecs start_time: 1784702288.706689926 secs.nsecs elapsed_time: 209.283959289 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 11 11 4: 0 0 11 8: 3 33 44 16: 1 11 55 32: 1 11 66 64: 0 0 66 128: 1 11 77 256: 2 22 100 - open/close 1420 (time 1784702509.28 total 10.01 last 141.92) - open/close 2785 (time 1784702519.28 total 20.01 last 136.50) - open/close 4163 (time 1784702529.28 total 30.01 last 137.77) - open/close 5701 (time 1784702539.28 total 40.01 last 153.79) - open/close 7399 (time 1784702549.28 total 50.01 last 169.79) - open/close 9016 (time 1784702559.28 total 60.01 last 161.69) total: 10000 open/close in 66.27 seconds: 150.90 ops/second llite.lustre-ffff8b8d47221800.statahead_batch_max=0 llite.lustre-ffff8b8d47221800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b8d47221800.stats=clear mdc.lustre-MDT0001-mdc-ffff8b8d47221800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m34.534s user 0m0.243s sys 0m27.623s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b8d47221800.statahead_batch_max=256 llite.lustre-ffff8b8d47221800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b8d47221800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8b8d47221800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b8d47221800.stats=clear mdc.lustre-MDT0001-mdc-ffff8b8d47221800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.206s user 0m0.200s sys 0m21.292s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8b8d47221800.batch_stats= snapshot_time: 1784702641.672318001 secs.nsecs start_time: 1784702604.110059199 secs.nsecs elapsed_time: 37.562258802 secs.nsecs subreqs per batch batches %% cum %% 1: 348 62 62 2: 93 16 78 4: 46 8 87 8: 9 1 88 16: 6 1 89 32: 19 3 93 64: 4 0 93 128: 0 0 93 256: 34 6 100 mdc.lustre-MDT0001-mdc-ffff8b8d47221800.batch_stats= snapshot_time: 1784702641.672375748 secs.nsecs start_time: 1784702604.110125222 secs.nsecs elapsed_time: 37.562250526 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8b8d47221800.statahead_batch_max=64 llite.lustre-ffff8b8d47221800.statahead_max=128