== sanity test 123ad: Verify batching statahead works correctly ========================================================== 17:48:21 (1781214501) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9dd043aae000.batch_stats=0 llite.lustre-ffff9dd043aae000.statahead_max=32 llite.lustre-ffff9dd043aae000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9dd043aae000.batch_stats=0 total: 100 open/close in 3.20 seconds: 31.23 ops/second llite.lustre-ffff9dd043aae000.statahead_max=0 101 real 0m2.348s user 0m0.003s sys 0m0.335s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9dd043aae000.statahead_max=32 llite.lustre-ffff9dd043aae000.statahead_max=32 101 real 0m1.441s user 0m0.008s sys 0m0.196s 'ls -l' 100 files with statahead: 2 sec statahead total: 169 statahead wrong: 2 agl total: 162 list_total: 30 fname_total: 132 hit_total: 4060 miss_total: 1565 swrong=2 ewrong=2 - open/close 624 (time 1781214532.63 total 10.01 last 62.36) total: 900 open/close in 14.60 seconds: 61.65 ops/second llite.lustre-ffff9dd043aae000.statahead_max=0 1001 real 0m28.376s user 0m0.045s sys 0m3.618s 'ls -l' 1000 files without statahead: 29 sec llite.lustre-ffff9dd043aae000.statahead_max=32 llite.lustre-ffff9dd043aae000.statahead_max=32 1001 real 0m11.555s user 0m0.035s sys 0m1.475s 'ls -l' 1000 files with statahead: 12 sec statahead total: 170 statahead wrong: 2 agl total: 163 list_total: 31 fname_total: 132 hit_total: 5058 miss_total: 1567 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff9dd043aae000.statahead_stats= statahead total: 170 statahead wrong: 2 agl total: 163 list_total: 31 fname_total: 132 hit_total: 5058 miss_total: 1567 mdc.lustre-MDT0000-mdc-ffff9dd043aae000.batch_stats= snapshot_time: 1781214620.534583186 secs.nsecs start_time: 1781214506.993759464 secs.nsecs elapsed_time: 113.540823722 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 598 (time 1781214633.73 total 10.02 last 59.71) - open/close 1273 (time 1781214643.73 total 20.02 last 67.49) - open/close 1902 (time 1781214653.74 total 30.02 last 62.86) - open/close 2481 (time 1781214663.75 total 40.03 last 57.86) - open/close 2933 (time 1781214673.75 total 50.03 last 45.19) - open/close 3494 (time 1781214683.75 total 60.03 last 56.09) - open/close 4042 (time 1781214693.77 total 70.05 last 54.71) - open/close 4649 (time 1781214703.77 total 80.05 last 60.68) - open/close 5246 (time 1781214713.79 total 90.07 last 59.59) - open/close 5787 (time 1781214723.79 total 100.08 last 54.06) - open/close 6232 (time 1781214733.80 total 110.08 last 44.49) - open/close 6575 (time 1781214743.80 total 120.09 last 34.28) - open/close 6997 (time 1781214753.81 total 130.09 last 42.19) - open/close 7511 (time 1781214763.83 total 140.11 last 51.30) - open/close 8027 (time 1781214773.83 total 150.11 last 51.57) - open/close 8567 (time 1781214783.84 total 160.12 last 53.97) - open/close 9011 (time 1781214793.86 total 170.14 last 44.31) - open/close 9645 (time 1781214803.90 total 180.18 last 63.14) total: 10000 open/close in 185.80 seconds: 53.82 ops/second llite.lustre-ffff9dd043aae000.statahead_batch_max=0 llite.lustre-ffff9dd043aae000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9dd043aae000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m24.412s user 0m0.414s sys 0m17.303s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9dd043aae000.statahead_batch_max=32 llite.lustre-ffff9dd043aae000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9dd043aae000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9dd043aae000.stats=clear ls -l files in d123ad.sanity: 10001 real 2m0.093s user 0m0.373s sys 0m16.096s batched statahead RPC= hit count=9983 mdc.lustre-MDT0000-mdc-ffff9dd043aae000.batch_stats= snapshot_time: 1781215040.816778540 secs.nsecs start_time: 1781214905.455732107 secs.nsecs elapsed_time: 135.361046433 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9dd043aae000.batch_stats=0 llite.lustre-ffff9dd043aae000.statahead_max=2048 llite.lustre-ffff9dd043aae000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9dd043aae000.batch_stats=0 total: 100 open/close in 3.37 seconds: 29.66 ops/second llite.lustre-ffff9dd043aae000.statahead_max=0 101 real 0m2.949s user 0m0.006s sys 0m0.361s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9dd043aae000.statahead_max=2048 llite.lustre-ffff9dd043aae000.statahead_max=2048 101 real 0m1.606s user 0m0.007s sys 0m0.189s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10082 miss_total: 18 swrong=0 ewrong=0 - open/close 514 (time 1781215427.90 total 10.06 last 51.08) total: 900 open/close in 16.17 seconds: 55.65 ops/second llite.lustre-ffff9dd043aae000.statahead_max=0 1001 real 0m31.982s user 0m0.046s sys 0m3.647s 'ls -l' 1000 files without statahead: 32 sec llite.lustre-ffff9dd043aae000.statahead_max=2048 llite.lustre-ffff9dd043aae000.statahead_max=2048 1001 real 0m10.262s user 0m0.042s sys 0m1.538s 'ls -l' 1000 files with statahead: 11 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff9dd043aae000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 mdc.lustre-MDT0000-mdc-ffff9dd043aae000.batch_stats= snapshot_time: 1781215522.309414613 secs.nsecs start_time: 1781215043.851806021 secs.nsecs elapsed_time: 478.457608592 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 581 (time 1781215535.75 total 10.02 last 57.97) - open/close 1054 (time 1781215545.75 total 20.02 last 47.29) - open/close 1649 (time 1781215555.75 total 30.03 last 59.48) - open/close 2361 (time 1781215565.76 total 40.03 last 71.17) - open/close 3046 (time 1781215575.77 total 50.04 last 68.45) - open/close 3654 (time 1781215585.78 total 60.05 last 60.72) - open/close 4270 (time 1781215595.79 total 70.06 last 61.54) - open/close 4848 (time 1781215605.80 total 80.07 last 57.75) - open/close 5373 (time 1781215615.81 total 90.08 last 52.44) - open/close 5916 (time 1781215625.81 total 100.09 last 54.27) - open/close 6546 (time 1781215635.82 total 110.09 last 62.99) - open/close 7041 (time 1781215645.84 total 120.11 last 49.40) - open/close 7660 (time 1781215655.85 total 130.13 last 61.79) - open/close 8337 (time 1781215665.87 total 140.14 last 67.63) - open/close 8953 (time 1781215675.88 total 150.15 last 61.53) - open/close 9339 (time 1781215685.90 total 160.17 last 38.53) - open/close 9724 (time 1781215695.90 total 170.17 last 38.49) total: 10000 open/close in 174.90 seconds: 57.18 ops/second llite.lustre-ffff9dd043aae000.statahead_batch_max=0 llite.lustre-ffff9dd043aae000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9dd043aae000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m16.802s user 0m0.372s sys 0m16.340s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9dd043aae000.statahead_batch_max=256 llite.lustre-ffff9dd043aae000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9dd043aae000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9dd043aae000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.877s user 0m0.378s sys 0m16.917s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9dd043aae000.batch_stats= snapshot_time: 1781215879.916044557 secs.nsecs start_time: 1781215790.045041102 secs.nsecs elapsed_time: 89.871003455 secs.nsecs subreqs per batch batches % cum % 1: 176 49 49 2: 61 17 67 4: 29 8 75 8: 18 5 80 16: 16 4 84 32: 15 4 89 64: 3 0 90 128: 0 0 90 256: 35 9 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9dd043aae000.statahead_batch_max=64 llite.lustre-ffff9dd043aae000.statahead_max=128