== sanity test 123ad: Verify batching statahead works correctly ========================================================== 18:49:13 (1788648553) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff98cbf9748800.batch_stats=0 llite.lustre-ffff98cbf9748800.statahead_max=32 llite.lustre-ffff98cbf9748800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff98cbf9748800.batch_stats=0 total: 100 open/close in 2.15 seconds: 46.45 ops/second llite.lustre-ffff98cbf9748800.statahead_max=0 101 real 0m1.842s user 0m0.005s sys 0m0.306s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff98cbf9748800.statahead_max=32 llite.lustre-ffff98cbf9748800.statahead_max=32 101 real 0m0.951s user 0m0.003s sys 0m0.163s 'ls -l' 100 files with statahead: 0 sec statahead total: 113 statahead wrong: 2 agl total: 107 list_total: 29 fname_total: 79 hit_total: 3818 miss_total: 1519 swrong=2 ewrong=2 - open/close 704 (time 1788648579.50 total 10.01 last 70.36) total: 900 open/close in 13.31 seconds: 67.61 ops/second llite.lustre-ffff98cbf9748800.statahead_max=0 1001 real 0m21.309s user 0m0.040s sys 0m2.882s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff98cbf9748800.statahead_max=32 llite.lustre-ffff98cbf9748800.statahead_max=32 1001 real 0m9.718s user 0m0.033s sys 0m1.349s 'ls -l' 1000 files with statahead: 9 sec statahead total: 114 statahead wrong: 2 agl total: 108 list_total: 30 fname_total: 79 hit_total: 4816 miss_total: 1521 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff98cbf9748800.statahead_stats= statahead total: 114 statahead wrong: 2 agl total: 108 list_total: 30 fname_total: 79 hit_total: 4816 miss_total: 1521 mdc.lustre-MDT0000-mdc-ffff98cbf9748800.batch_stats= snapshot_time: 1788648645.432451541 secs.nsecs start_time: 1788648557.500318322 secs.nsecs elapsed_time: 87.932133219 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 751 (time 1788648657.34 total 10.01 last 75.02) - open/close 1455 (time 1788648667.37 total 20.04 last 70.20) - open/close 2180 (time 1788648677.37 total 30.04 last 72.48) - open/close 2864 (time 1788648687.38 total 40.05 last 68.34) - open/close 3662 (time 1788648697.39 total 50.05 last 79.77) - open/close 4374 (time 1788648707.39 total 60.06 last 71.16) - open/close 5070 (time 1788648717.40 total 70.07 last 69.55) - open/close 5650 (time 1788648727.41 total 80.08 last 57.92) - open/close 6142 (time 1788648737.44 total 90.11 last 49.07) - open/close 6827 (time 1788648747.45 total 100.12 last 68.43) - open/close 7627 (time 1788648757.45 total 110.12 last 80.00) - open/close 8339 (time 1788648767.46 total 120.12 last 71.16) - open/close 9037 (time 1788648777.46 total 130.13 last 69.78) - open/close 9723 (time 1788648787.48 total 140.14 last 68.49) total: 10000 open/close in 144.12 seconds: 69.39 ops/second llite.lustre-ffff98cbf9748800.statahead_batch_max=0 llite.lustre-ffff98cbf9748800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98cbf9748800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.889s user 0m0.316s sys 0m14.609s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff98cbf9748800.statahead_batch_max=32 llite.lustre-ffff98cbf9748800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98cbf9748800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff98cbf9748800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m37.299s user 0m0.310s sys 0m13.495s batched statahead RPC= hit count=9971 mdc.lustre-MDT0000-mdc-ffff98cbf9748800.batch_stats= snapshot_time: 1788648972.643341615 secs.nsecs start_time: 1788648865.976883089 secs.nsecs elapsed_time: 106.666458526 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff98cbf9748800.batch_stats=0 llite.lustre-ffff98cbf9748800.statahead_max=2048 llite.lustre-ffff98cbf9748800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff98cbf9748800.batch_stats=0 total: 100 open/close in 3.67 seconds: 27.26 ops/second llite.lustre-ffff98cbf9748800.statahead_max=0 101 real 0m3.207s user 0m0.008s sys 0m0.325s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff98cbf9748800.statahead_max=2048 llite.lustre-ffff98cbf9748800.statahead_max=2048 101 real 0m0.969s user 0m0.007s sys 0m0.166s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10070 miss_total: 30 swrong=0 ewrong=0 - open/close 786 (time 1788649298.49 total 10.00 last 78.58) total: 900 open/close in 11.77 seconds: 76.45 ops/second llite.lustre-ffff98cbf9748800.statahead_max=0 1001 real 0m19.952s user 0m0.034s sys 0m2.840s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff98cbf9748800.statahead_max=2048 llite.lustre-ffff98cbf9748800.statahead_max=2048 1001 real 0m6.821s user 0m0.028s sys 0m1.320s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11069 miss_total: 31 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff98cbf9748800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11069 miss_total: 31 mdc.lustre-MDT0000-mdc-ffff98cbf9748800.batch_stats= snapshot_time: 1788649359.066051634 secs.nsecs start_time: 1788648975.177670171 secs.nsecs elapsed_time: 383.888381463 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 3 25 58 64: 1 8 66 128: 0 0 66 256: 4 33 100 - open/close 636 (time 1788649371.63 total 10.00 last 63.57) - open/close 1380 (time 1788649381.63 total 20.01 last 74.37) - open/close 2097 (time 1788649391.64 total 30.01 last 71.69) - open/close 2748 (time 1788649401.65 total 40.02 last 65.00) - open/close 3407 (time 1788649411.66 total 50.03 last 65.86) - open/close 3978 (time 1788649421.67 total 60.04 last 57.04) - open/close 4648 (time 1788649431.67 total 70.05 last 66.95) - open/close 5205 (time 1788649441.68 total 80.05 last 55.67) - open/close 5974 (time 1788649451.69 total 90.06 last 76.85) - open/close 6584 (time 1788649461.69 total 100.07 last 60.97) - open/close 7217 (time 1788649471.70 total 110.07 last 63.27) - open/close 7824 (time 1788649481.70 total 120.08 last 60.66) - open/close 8383 (time 1788649491.72 total 130.09 last 55.82) - open/close 8904 (time 1788649501.73 total 140.11 last 52.02) - open/close 9395 (time 1788649511.74 total 150.11 last 49.09) total: 10000 open/close in 159.95 seconds: 62.52 ops/second llite.lustre-ffff98cbf9748800.statahead_batch_max=0 llite.lustre-ffff98cbf9748800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98cbf9748800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m59.596s user 0m0.373s sys 0m15.447s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff98cbf9748800.statahead_batch_max=256 llite.lustre-ffff98cbf9748800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98cbf9748800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff98cbf9748800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m48.897s user 0m0.332s sys 0m15.798s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff98cbf9748800.batch_stats= snapshot_time: 1788649646.939488966 secs.nsecs start_time: 1788649588.208999865 secs.nsecs elapsed_time: 58.730489101 secs.nsecs subreqs per batch batches %% cum %% 1: 154 53 53 2: 28 9 63 4: 27 9 72 8: 11 3 76 16: 12 4 80 32: 17 5 86 64: 4 1 87 128: 0 0 87 256: 35 12 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff98cbf9748800.statahead_batch_max=64 llite.lustre-ffff98cbf9748800.statahead_max=128