== sanity test 123ad: Verify batching statahead works correctly ========================================================== 07:55:18 (1788782118) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff93850ccd0800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93850ccd0800.batch_stats=0 llite.lustre-ffff93850ccd0800.statahead_max=32 llite.lustre-ffff93850ccd0800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff93850ccd0800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93850ccd0800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 3.15 seconds: 31.74 ops/second llite.lustre-ffff93850ccd0800.statahead_max=0 101 real 0m2.551s user 0m0.006s sys 0m0.339s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff93850ccd0800.statahead_max=32 llite.lustre-ffff93850ccd0800.statahead_max=32 101 real 0m1.591s user 0m0.006s sys 0m0.162s 'ls -l' 100 files with statahead: 2 sec statahead total: 130 statahead wrong: 2 agl total: 124 list_total: 31 fname_total: 83 hit_total: 4119 miss_total: 1563 swrong=2 ewrong=2 - open/close 626 (time 1788782149.02 total 10.02 last 62.46) total: 900 open/close in 14.54 seconds: 61.89 ops/second llite.lustre-ffff93850ccd0800.statahead_max=0 1001 real 0m33.198s user 0m0.038s sys 0m3.353s 'ls -l' 1000 files without statahead: 33 sec llite.lustre-ffff93850ccd0800.statahead_max=32 llite.lustre-ffff93850ccd0800.statahead_max=32 1001 real 0m11.115s user 0m0.033s sys 0m1.512s 'ls -l' 1000 files with statahead: 11 sec statahead total: 131 statahead wrong: 2 agl total: 125 list_total: 32 fname_total: 83 hit_total: 5117 miss_total: 1565 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff93850ccd0800.statahead_stats= statahead total: 131 statahead wrong: 2 agl total: 125 list_total: 32 fname_total: 83 hit_total: 5117 miss_total: 1565 mdc.lustre-MDT0000-mdc-ffff93850ccd0800.batch_stats= snapshot_time: 1788782237.881099493 secs.nsecs start_time: 1788782123.623625844 secs.nsecs elapsed_time: 114.257473649 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 2 5 7 4: 2 5 12 8: 0 0 12 16: 18 46 58 32: 16 41 100 mdc.lustre-MDT0001-mdc-ffff93850ccd0800.batch_stats= snapshot_time: 1788782237.881162625 secs.nsecs start_time: 1788782123.623781919 secs.nsecs elapsed_time: 114.257380706 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 1 2 5 4: 0 0 5 8: 4 10 15 16: 19 48 64 32: 14 35 100 - open/close 634 (time 1788782250.93 total 10.01 last 63.31) - open/close 1288 (time 1788782260.94 total 20.02 last 65.37) - open/close 1994 (time 1788782270.94 total 30.03 last 70.55) - open/close 2634 (time 1788782280.95 total 40.03 last 63.96) - open/close 3287 (time 1788782290.95 total 50.04 last 65.29) - open/close 4020 (time 1788782300.96 total 60.04 last 73.26) - open/close 4680 (time 1788782310.96 total 70.05 last 65.96) - open/close 5259 (time 1788782320.98 total 80.07 last 57.78) - open/close 5863 (time 1788782330.99 total 90.07 last 60.39) - open/close 6524 (time 1788782340.99 total 100.08 last 66.07) - open/close 7200 (time 1788782351.00 total 110.08 last 67.56) - open/close 7819 (time 1788782361.00 total 120.09 last 61.85) - open/close 8523 (time 1788782371.02 total 130.10 last 70.29) - open/close 9209 (time 1788782381.04 total 140.12 last 68.48) - open/close 9828 (time 1788782391.05 total 150.14 last 61.81) total: 10000 open/close in 152.88 seconds: 65.41 ops/second llite.lustre-ffff93850ccd0800.statahead_batch_max=0 llite.lustre-ffff93850ccd0800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93850ccd0800.stats=clear mdc.lustre-MDT0001-mdc-ffff93850ccd0800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m19.463s user 0m0.416s sys 0m17.052s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93850ccd0800.statahead_batch_max=32 llite.lustre-ffff93850ccd0800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93850ccd0800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff93850ccd0800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93850ccd0800.stats=clear mdc.lustre-MDT0001-mdc-ffff93850ccd0800.stats=clear ls -l files in d123ad.sanity: 10001 real 2m15.743s user 0m0.342s sys 0m15.159s batched statahead RPC= hit count=9987 mdc.lustre-MDT0000-mdc-ffff93850ccd0800.batch_stats= snapshot_time: 1788782634.585448303 secs.nsecs start_time: 1788782485.613975163 secs.nsecs elapsed_time: 148.971473140 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff93850ccd0800.batch_stats= snapshot_time: 1788782634.585519230 secs.nsecs start_time: 1788782485.614094864 secs.nsecs elapsed_time: 148.971424366 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff93850ccd0800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93850ccd0800.batch_stats=0 llite.lustre-ffff93850ccd0800.statahead_max=2048 llite.lustre-ffff93850ccd0800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff93850ccd0800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93850ccd0800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 3.02 seconds: 33.15 ops/second llite.lustre-ffff93850ccd0800.statahead_max=0 101 real 0m3.581s user 0m0.004s sys 0m0.348s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff93850ccd0800.statahead_max=2048 llite.lustre-ffff93850ccd0800.statahead_max=2048 101 real 0m0.991s user 0m0.003s sys 0m0.180s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10086 miss_total: 14 swrong=0 ewrong=0 - open/close 765 (time 1788782987.02 total 10.00 last 76.48) total: 900 open/close in 11.62 seconds: 77.43 ops/second llite.lustre-ffff93850ccd0800.statahead_max=0 1001 real 0m27.128s user 0m0.047s sys 0m3.120s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff93850ccd0800.statahead_max=2048 llite.lustre-ffff93850ccd0800.statahead_max=2048 1001 real 0m7.690s user 0m0.023s sys 0m1.476s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff93850ccd0800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 mdc.lustre-MDT0000-mdc-ffff93850ccd0800.batch_stats= snapshot_time: 1788783059.187926369 secs.nsecs start_time: 1788782637.830695379 secs.nsecs elapsed_time: 421.357230990 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 2 22 22 4: 2 22 44 8: 0 0 44 16: 2 22 66 32: 1 11 77 64: 0 0 77 128: 0 0 77 256: 2 22 100 mdc.lustre-MDT0001-mdc-ffff93850ccd0800.batch_stats= snapshot_time: 1788783059.188005397 secs.nsecs start_time: 1788782637.830877494 secs.nsecs elapsed_time: 421.357127903 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 5 55 55 16: 0 0 55 32: 2 22 77 64: 0 0 77 128: 0 0 77 256: 2 22 100 - open/close 540 (time 1788783072.41 total 10.01 last 53.93) - open/close 1103 (time 1788783082.41 total 20.02 last 56.29) - open/close 1655 (time 1788783092.42 total 30.03 last 55.15) - open/close 2251 (time 1788783102.44 total 40.04 last 59.50) - open/close 2863 (time 1788783112.45 total 50.05 last 61.13) - open/close 3589 (time 1788783122.47 total 60.07 last 72.49) - open/close 4293 (time 1788783132.47 total 70.07 last 70.37) - open/close 4995 (time 1788783142.47 total 80.08 last 70.19) - open/close 5763 (time 1788783152.48 total 90.08 last 76.76) - open/close 6518 (time 1788783162.49 total 100.09 last 75.40) - open/close 7272 (time 1788783172.50 total 110.10 last 75.32) - open/close 8009 (time 1788783182.51 total 120.11 last 73.64) - open/close 8734 (time 1788783192.52 total 130.12 last 72.45) - open/close 9423 (time 1788783202.52 total 140.12 last 68.89) total: 10000 open/close in 147.92 seconds: 67.61 ops/second llite.lustre-ffff93850ccd0800.statahead_batch_max=0 llite.lustre-ffff93850ccd0800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93850ccd0800.stats=clear mdc.lustre-MDT0001-mdc-ffff93850ccd0800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.858s user 0m0.393s sys 0m17.351s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93850ccd0800.statahead_batch_max=256 llite.lustre-ffff93850ccd0800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93850ccd0800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff93850ccd0800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93850ccd0800.stats=clear mdc.lustre-MDT0001-mdc-ffff93850ccd0800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.785s user 0m0.344s sys 0m16.532s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff93850ccd0800.batch_stats= snapshot_time: 1788783354.073716648 secs.nsecs start_time: 1788783286.681689855 secs.nsecs elapsed_time: 67.392026793 secs.nsecs subreqs per batch batches %% cum %% 1: 49 32 32 2: 12 8 40 4: 17 11 52 8: 6 4 56 16: 7 4 60 32: 13 8 69 64: 10 6 76 128: 0 0 76 256: 36 24 100 mdc.lustre-MDT0001-mdc-ffff93850ccd0800.batch_stats= snapshot_time: 1788783354.073934454 secs.nsecs start_time: 1788783286.681779550 secs.nsecs elapsed_time: 67.392154904 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff93850ccd0800.statahead_batch_max=64 llite.lustre-ffff93850ccd0800.statahead_max=128