== sanity test 123ad: Verify batching statahead works correctly ========================================================== 05:38:03 (1788514683) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff98178635d000.batch_stats=0 llite.lustre-ffff98178635d000.statahead_max=32 llite.lustre-ffff98178635d000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff98178635d000.batch_stats=0 total: 100 open/close in 3.00 seconds: 33.35 ops/second llite.lustre-ffff98178635d000.statahead_max=0 101 real 0m2.679s user 0m0.005s sys 0m0.347s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff98178635d000.statahead_max=32 llite.lustre-ffff98178635d000.statahead_max=32 101 real 0m1.133s user 0m0.006s sys 0m0.171s 'ls -l' 100 files with statahead: 1 sec statahead total: 133 statahead wrong: 2 agl total: 127 list_total: 30 fname_total: 88 hit_total: 3963 miss_total: 1725 swrong=2 ewrong=2 - open/close 656 (time 1788514711.86 total 10.00 last 65.60) total: 900 open/close in 14.36 seconds: 62.66 ops/second llite.lustre-ffff98178635d000.statahead_max=0 1001 real 0m25.481s user 0m0.028s sys 0m3.000s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff98178635d000.statahead_max=32 llite.lustre-ffff98178635d000.statahead_max=32 1001 real 0m10.946s user 0m0.021s sys 0m1.612s 'ls -l' 1000 files with statahead: 11 sec statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 31 fname_total: 88 hit_total: 4962 miss_total: 1726 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff98178635d000.statahead_stats= statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 31 fname_total: 88 hit_total: 4962 miss_total: 1726 mdc.lustre-MDT0000-mdc-ffff98178635d000.batch_stats= snapshot_time: 1788514790.097223105 secs.nsecs start_time: 1788514687.928597991 secs.nsecs elapsed_time: 102.168625114 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 1 2 5 8: 2 5 10 16: 2 5 15 32: 34 85 100 - open/close 637 (time 1788514803.01 total 10.01 last 63.66) - open/close 1338 (time 1788514813.02 total 20.01 last 70.07) - open/close 2012 (time 1788514823.03 total 30.02 last 67.33) - open/close 2663 (time 1788514833.03 total 40.02 last 65.09) - open/close 3397 (time 1788514843.04 total 50.03 last 73.31) - open/close 4027 (time 1788514853.05 total 60.04 last 62.95) - open/close 4676 (time 1788514863.06 total 70.05 last 64.82) - open/close 5328 (time 1788514873.07 total 80.06 last 65.15) - open/close 6015 (time 1788514883.10 total 90.10 last 68.45) - open/close 6699 (time 1788514893.11 total 100.10 last 68.37) - open/close 7423 (time 1788514903.11 total 110.11 last 72.37) - open/close 8084 (time 1788514913.11 total 120.11 last 66.09) - open/close 8751 (time 1788514923.13 total 130.12 last 66.60) - open/close 9373 (time 1788514933.13 total 140.13 last 62.17) - open/close 9967 (time 1788514943.17 total 150.17 last 59.17) total: 10000 open/close in 150.67 seconds: 66.37 ops/second llite.lustre-ffff98178635d000.statahead_batch_max=0 llite.lustre-ffff98178635d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98178635d000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.324s user 0m0.321s sys 0m17.857s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff98178635d000.statahead_batch_max=32 llite.lustre-ffff98178635d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98178635d000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff98178635d000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m44.294s user 0m0.292s sys 0m15.821s batched statahead RPC= hit count=9986 mdc.lustre-MDT0000-mdc-ffff98178635d000.batch_stats= snapshot_time: 1788515144.367351056 secs.nsecs start_time: 1788515024.932243749 secs.nsecs elapsed_time: 119.435107307 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff98178635d000.batch_stats=0 llite.lustre-ffff98178635d000.statahead_max=2048 llite.lustre-ffff98178635d000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff98178635d000.batch_stats=0 total: 100 open/close in 2.84 seconds: 35.19 ops/second llite.lustre-ffff98178635d000.statahead_max=0 101 real 0m2.634s user 0m0.005s sys 0m0.342s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff98178635d000.statahead_max=2048 llite.lustre-ffff98178635d000.statahead_max=2048 101 real 0m1.051s user 0m0.006s sys 0m0.183s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10085 miss_total: 15 swrong=0 ewrong=0 - open/close 695 (time 1788515478.19 total 10.01 last 69.44) total: 900 open/close in 13.02 seconds: 69.14 ops/second llite.lustre-ffff98178635d000.statahead_max=0 1001 real 0m23.188s user 0m0.033s sys 0m3.444s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff98178635d000.statahead_max=2048 llite.lustre-ffff98178635d000.statahead_max=2048 1001 real 0m7.533s user 0m0.041s sys 0m1.895s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff98178635d000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 mdc.lustre-MDT0000-mdc-ffff98178635d000.batch_stats= snapshot_time: 1788515550.271352255 secs.nsecs start_time: 1788515146.369513220 secs.nsecs elapsed_time: 403.901839035 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 1 8 41 64: 1 8 50 128: 3 25 75 256: 3 25 100 - open/close 643 (time 1788515563.36 total 10.02 last 64.17) - open/close 1210 (time 1788515573.37 total 20.03 last 56.67) - open/close 1823 (time 1788515583.38 total 30.03 last 61.24) - open/close 2409 (time 1788515593.39 total 40.05 last 58.53) - open/close 3040 (time 1788515603.40 total 50.06 last 63.02) - open/close 3699 (time 1788515613.41 total 60.07 last 65.83) - open/close 4330 (time 1788515623.42 total 70.07 last 63.08) - open/close 4970 (time 1788515633.42 total 80.08 last 63.98) - open/close 5592 (time 1788515643.44 total 90.10 last 62.07) - open/close 6252 (time 1788515653.45 total 100.11 last 65.91) - open/close 6942 (time 1788515663.46 total 110.12 last 68.95) - open/close 7523 (time 1788515673.49 total 120.15 last 57.93) - open/close 8198 (time 1788515683.50 total 130.16 last 67.43) - open/close 8809 (time 1788515693.50 total 140.16 last 61.10) - open/close 9425 (time 1788515703.50 total 150.16 last 61.59) - open/close 9996 (time 1788515713.50 total 160.16 last 57.09) total: 10000 open/close in 160.20 seconds: 62.42 ops/second llite.lustre-ffff98178635d000.statahead_batch_max=0 llite.lustre-ffff98178635d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98178635d000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m5.603s user 0m0.329s sys 0m17.914s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff98178635d000.statahead_batch_max=256 llite.lustre-ffff98178635d000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98178635d000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff98178635d000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.639s user 0m0.326s sys 0m16.698s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff98178635d000.batch_stats= snapshot_time: 1788515864.117761938 secs.nsecs start_time: 1788515790.555768885 secs.nsecs elapsed_time: 73.561993053 secs.nsecs subreqs per batch batches %% cum %% 1: 148 52 52 2: 35 12 65 4: 18 6 71 8: 14 4 76 16: 11 3 80 32: 14 4 85 64: 6 2 87 128: 0 0 87 256: 35 12 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff98178635d000.statahead_batch_max=64 llite.lustre-ffff98178635d000.statahead_max=128