== sanity test 123ad: Verify batching statahead works correctly ========================================================== 12:57:16 (1790960236) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff94cf42c08800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff94cf42c08800.batch_stats=0 llite.lustre-ffff94cf42c08800.statahead_max=32 llite.lustre-ffff94cf42c08800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff94cf42c08800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff94cf42c08800.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.89 seconds: 53.02 ops/second llite.lustre-ffff94cf42c08800.statahead_max=0 101 real 0m1.811s user 0m0.004s sys 0m0.298s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff94cf42c08800.statahead_max=32 llite.lustre-ffff94cf42c08800.statahead_max=32 101 real 0m0.861s user 0m0.004s sys 0m0.182s 'ls -l' 100 files with statahead: 1 sec statahead total: 102 statahead wrong: 2 agl total: 97 list_total: 32 fname_total: 64 hit_total: 4596 miss_total: 1289 swrong=2 ewrong=2 total: 900 open/close in 7.59 seconds: 118.65 ops/second llite.lustre-ffff94cf42c08800.statahead_max=0 1001 real 0m15.955s user 0m0.023s sys 0m2.702s 'ls -l' 1000 files without statahead: 16 sec llite.lustre-ffff94cf42c08800.statahead_max=32 llite.lustre-ffff94cf42c08800.statahead_max=32 1001 real 0m5.768s user 0m0.019s sys 0m1.391s 'ls -l' 1000 files with statahead: 6 sec statahead total: 103 statahead wrong: 2 agl total: 98 list_total: 33 fname_total: 64 hit_total: 5593 miss_total: 1292 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 14 seconds llite.lustre-ffff94cf42c08800.statahead_stats= statahead total: 103 statahead wrong: 2 agl total: 98 list_total: 33 fname_total: 64 hit_total: 5593 miss_total: 1292 mdc.lustre-MDT0000-mdc-ffff94cf42c08800.batch_stats= snapshot_time: 1790960299.326297171 secs.nsecs start_time: 1790960239.818628162 secs.nsecs elapsed_time: 59.507669009 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 4 11 13 8: 1 2 16 16: 26 72 88 32: 4 11 100 mdc.lustre-MDT0001-mdc-ffff94cf42c08800.batch_stats= snapshot_time: 1790960299.326359729 secs.nsecs start_time: 1790960239.818710967 secs.nsecs elapsed_time: 59.507648762 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 1 2 5 8: 3 7 12 16: 13 32 45 32: 22 55 100 - open/close 1067 (time 1790960311.39 total 10.00 last 106.68) - open/close 2153 (time 1790960321.39 total 20.00 last 108.57) - open/close 3173 (time 1790960331.40 total 30.01 last 101.99) - open/close 4183 (time 1790960341.40 total 40.01 last 100.99) - open/close 5218 (time 1790960351.40 total 50.01 last 103.43) - open/close 6261 (time 1790960361.41 total 60.02 last 104.28) - open/close 7234 (time 1790960371.41 total 70.02 last 97.30) - open/close 8156 (time 1790960381.41 total 80.02 last 92.20) - open/close 9157 (time 1790960391.41 total 90.02 last 100.06) total: 10000 open/close in 98.53 seconds: 101.49 ops/second llite.lustre-ffff94cf42c08800.statahead_batch_max=0 llite.lustre-ffff94cf42c08800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff94cf42c08800.stats=clear mdc.lustre-MDT0001-mdc-ffff94cf42c08800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m39.562s user 0m0.254s sys 0m19.145s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff94cf42c08800.statahead_batch_max=32 llite.lustre-ffff94cf42c08800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff94cf42c08800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff94cf42c08800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff94cf42c08800.stats=clear mdc.lustre-MDT0001-mdc-ffff94cf42c08800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m48.660s user 0m0.209s sys 0m16.510s batched statahead RPC= hit count=9985 mdc.lustre-MDT0000-mdc-ffff94cf42c08800.batch_stats= snapshot_time: 1790960499.897096898 secs.nsecs start_time: 1790960444.037764698 secs.nsecs elapsed_time: 55.859332200 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff94cf42c08800.batch_stats= snapshot_time: 1790960499.897154496 secs.nsecs start_time: 1790960444.037988590 secs.nsecs elapsed_time: 55.859165906 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff94cf42c08800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff94cf42c08800.batch_stats=0 llite.lustre-ffff94cf42c08800.statahead_max=2048 llite.lustre-ffff94cf42c08800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff94cf42c08800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff94cf42c08800.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.13 seconds: 88.83 ops/second llite.lustre-ffff94cf42c08800.statahead_max=0 101 real 0m1.011s user 0m0.006s sys 0m0.286s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff94cf42c08800.statahead_max=2048 llite.lustre-ffff94cf42c08800.statahead_max=2048 101 real 0m0.376s user 0m0.003s sys 0m0.214s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10084 miss_total: 16 swrong=0 ewrong=0 total: 900 open/close in 3.79 seconds: 237.45 ops/second llite.lustre-ffff94cf42c08800.statahead_max=0 1001 real 0m9.134s user 0m0.028s sys 0m2.682s 'ls -l' 1000 files without statahead: 9 sec llite.lustre-ffff94cf42c08800.statahead_max=2048 llite.lustre-ffff94cf42c08800.statahead_max=2048 1001 real 0m3.414s user 0m0.019s sys 0m1.411s 'ls -l' 1000 files with statahead: 3 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 8 seconds llite.lustre-ffff94cf42c08800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 mdc.lustre-MDT0000-mdc-ffff94cf42c08800.batch_stats= snapshot_time: 1790960660.900486677 secs.nsecs start_time: 1790960500.902597108 secs.nsecs elapsed_time: 159.997889569 secs.nsecs subreqs per batch batches %% cum %% 1: 1 16 16 2: 0 0 16 4: 3 50 66 8: 0 0 66 16: 0 0 66 32: 0 0 66 64: 0 0 66 128: 0 0 66 256: 2 33 100 mdc.lustre-MDT0001-mdc-ffff94cf42c08800.batch_stats= snapshot_time: 1790960660.900575783 secs.nsecs start_time: 1790960500.902740711 secs.nsecs elapsed_time: 159.997835072 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 33 33 16: 2 22 55 32: 0 0 55 64: 2 22 77 128: 0 0 77 256: 2 22 100 - open/close 2409 (time 1790960672.04 total 10.00 last 240.83) - open/close 4468 (time 1790960682.04 total 20.01 last 205.78) - open/close 6582 (time 1790960692.05 total 30.01 last 211.38) - open/close 8672 (time 1790960702.05 total 40.02 last 208.87) total: 10000 open/close in 45.58 seconds: 219.39 ops/second llite.lustre-ffff94cf42c08800.statahead_batch_max=0 llite.lustre-ffff94cf42c08800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff94cf42c08800.stats=clear mdc.lustre-MDT0001-mdc-ffff94cf42c08800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.358s user 0m0.239s sys 0m25.890s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff94cf42c08800.statahead_batch_max=256 llite.lustre-ffff94cf42c08800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff94cf42c08800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff94cf42c08800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff94cf42c08800.stats=clear mdc.lustre-MDT0001-mdc-ffff94cf42c08800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m26.259s user 0m0.209s sys 0m19.786s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff94cf42c08800.batch_stats= snapshot_time: 1790960772.390232115 secs.nsecs start_time: 1790960741.231773302 secs.nsecs elapsed_time: 31.158458813 secs.nsecs subreqs per batch batches %% cum %% 1: 89 49 49 2: 13 7 56 4: 10 5 62 8: 4 2 64 16: 1 0 65 32: 19 10 75 64: 7 3 79 128: 1 0 80 256: 35 19 100 mdc.lustre-MDT0001-mdc-ffff94cf42c08800.batch_stats= snapshot_time: 1790960772.390295185 secs.nsecs start_time: 1790960741.231895650 secs.nsecs elapsed_time: 31.158399535 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff94cf42c08800.statahead_batch_max=64 llite.lustre-ffff94cf42c08800.statahead_max=128