== sanity test 123ad: Verify batching statahead works correctly ========================================================== 03:48:06 (1787384886) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9a34c538f800.batch_stats=0 llite.lustre-ffff9a34c538f800.statahead_max=32 llite.lustre-ffff9a34c538f800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9a34c538f800.batch_stats=0 total: 100 open/close in 3.28 seconds: 30.51 ops/second llite.lustre-ffff9a34c538f800.statahead_max=0 101 real 0m2.739s user 0m0.008s sys 0m0.323s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9a34c538f800.statahead_max=32 llite.lustre-ffff9a34c538f800.statahead_max=32 101 real 0m1.134s user 0m0.007s sys 0m0.200s 'ls -l' 100 files with statahead: 1 sec statahead total: 126 statahead wrong: 2 agl total: 121 list_total: 30 fname_total: 96 hit_total: 4522 miss_total: 1284 swrong=2 ewrong=2 - open/close 610 (time 1787384915.41 total 10.01 last 60.93) total: 900 open/close in 14.11 seconds: 63.79 ops/second llite.lustre-ffff9a34c538f800.statahead_max=0 1001 real 0m26.816s user 0m0.042s sys 0m3.344s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff9a34c538f800.statahead_max=32 llite.lustre-ffff9a34c538f800.statahead_max=32 1001 real 0m10.566s user 0m0.029s sys 0m1.639s 'ls -l' 1000 files with statahead: 11 sec statahead total: 127 statahead wrong: 2 agl total: 122 list_total: 31 fname_total: 96 hit_total: 5520 miss_total: 1286 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff9a34c538f800.statahead_stats= statahead total: 127 statahead wrong: 2 agl total: 122 list_total: 31 fname_total: 96 hit_total: 5520 miss_total: 1286 mdc.lustre-MDT0000-mdc-ffff9a34c538f800.batch_stats= snapshot_time: 1787384992.156731315 secs.nsecs start_time: 1787384892.067529635 secs.nsecs elapsed_time: 100.089201680 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 697 (time 1787385005.35 total 10.01 last 69.64) - open/close 1297 (time 1787385015.36 total 20.02 last 59.94) - open/close 1894 (time 1787385025.37 total 30.03 last 59.63) - open/close 2498 (time 1787385035.39 total 40.05 last 60.27) - open/close 3059 (time 1787385045.40 total 50.06 last 56.06) - open/close 3658 (time 1787385055.42 total 60.08 last 59.79) - open/close 4276 (time 1787385065.43 total 70.09 last 61.72) - open/close 4802 (time 1787385075.45 total 80.11 last 52.50) - open/close 5370 (time 1787385085.46 total 90.12 last 56.77) - open/close 5993 (time 1787385095.46 total 100.12 last 62.27) - open/close 6633 (time 1787385105.47 total 110.13 last 63.95) - open/close 7291 (time 1787385115.47 total 120.13 last 65.77) - open/close 7964 (time 1787385125.47 total 130.13 last 67.29) - open/close 8635 (time 1787385135.48 total 140.14 last 67.06) - open/close 9300 (time 1787385145.49 total 150.15 last 66.43) - open/close 9869 (time 1787385155.49 total 160.15 last 56.89) total: 10000 open/close in 162.37 seconds: 61.59 ops/second llite.lustre-ffff9a34c538f800.statahead_batch_max=0 llite.lustre-ffff9a34c538f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a34c538f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m11.686s user 0m0.338s sys 0m16.505s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9a34c538f800.statahead_batch_max=32 llite.lustre-ffff9a34c538f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a34c538f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9a34c538f800.stats=clear ls -l files in d123ad.sanity: 10001 real 2m10.285s user 0m0.324s sys 0m15.539s batched statahead RPC= hit count=9975 mdc.lustre-MDT0000-mdc-ffff9a34c538f800.batch_stats= snapshot_time: 1787385384.535651542 secs.nsecs start_time: 1787385240.017373287 secs.nsecs elapsed_time: 144.518278255 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9a34c538f800.batch_stats=0 llite.lustre-ffff9a34c538f800.statahead_max=2048 llite.lustre-ffff9a34c538f800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9a34c538f800.batch_stats=0 total: 100 open/close in 2.55 seconds: 39.17 ops/second llite.lustre-ffff9a34c538f800.statahead_max=0 101 real 0m2.910s user 0m0.006s sys 0m0.397s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9a34c538f800.statahead_max=2048 llite.lustre-ffff9a34c538f800.statahead_max=2048 101 real 0m1.253s user 0m0.006s sys 0m0.203s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10074 miss_total: 26 swrong=0 ewrong=0 - open/close 584 (time 1787385760.03 total 10.01 last 58.34) total: 900 open/close in 15.47 seconds: 58.16 ops/second llite.lustre-ffff9a34c538f800.statahead_max=0 1001 real 0m24.107s user 0m0.043s sys 0m3.505s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff9a34c538f800.statahead_max=2048 llite.lustre-ffff9a34c538f800.statahead_max=2048 1001 real 0m6.748s user 0m0.028s sys 0m1.541s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11073 miss_total: 27 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff9a34c538f800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11073 miss_total: 27 mdc.lustre-MDT0000-mdc-ffff9a34c538f800.batch_stats= snapshot_time: 1787385833.773461738 secs.nsecs start_time: 1787385387.323028028 secs.nsecs elapsed_time: 446.450433710 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 2 16 66 128: 0 0 66 256: 4 33 100 - open/close 632 (time 1787385846.07 total 10.01 last 63.17) - open/close 1251 (time 1787385856.07 total 20.01 last 61.88) - open/close 1899 (time 1787385866.08 total 30.02 last 64.75) - open/close 2542 (time 1787385876.09 total 40.03 last 64.22) - open/close 3197 (time 1787385886.10 total 50.03 last 65.47) - open/close 3919 (time 1787385896.11 total 60.04 last 72.13) - open/close 4550 (time 1787385906.12 total 70.06 last 63.02) - open/close 5160 (time 1787385916.12 total 80.06 last 60.99) - open/close 5742 (time 1787385926.14 total 90.07 last 58.10) - open/close 6312 (time 1787385936.15 total 100.09 last 56.90) - open/close 6954 (time 1787385946.16 total 110.10 last 64.14) - open/close 7599 (time 1787385956.17 total 120.11 last 64.42) - open/close 8208 (time 1787385966.19 total 130.12 last 60.83) - open/close 8757 (time 1787385976.19 total 140.13 last 54.90) - open/close 9301 (time 1787385986.22 total 150.16 last 54.20) - open/close 9900 (time 1787385996.23 total 160.16 last 59.89) total: 10000 open/close in 162.19 seconds: 61.66 ops/second llite.lustre-ffff9a34c538f800.statahead_batch_max=0 llite.lustre-ffff9a34c538f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a34c538f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.020s user 0m0.332s sys 0m16.575s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9a34c538f800.statahead_batch_max=256 llite.lustre-ffff9a34c538f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a34c538f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9a34c538f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.319s user 0m0.285s sys 0m15.872s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9a34c538f800.batch_stats= snapshot_time: 1787386157.835556228 secs.nsecs start_time: 1787386079.763318263 secs.nsecs elapsed_time: 78.072237965 secs.nsecs subreqs per batch batches %% cum %% 1: 250 55 55 2: 63 13 69 4: 44 9 78 8: 29 6 85 16: 15 3 88 32: 12 2 91 64: 5 1 92 128: 0 0 92 256: 34 7 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9a34c538f800.statahead_batch_max=64 llite.lustre-ffff9a34c538f800.statahead_max=128