== sanity test 123ad: Verify batching statahead works correctly ========================================================== 13:39:15 (1784050755) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff93d507c93800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93d507c93800.batch_stats=0 llite.lustre-ffff93d507c93800.statahead_max=32 llite.lustre-ffff93d507c93800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff93d507c93800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93d507c93800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.45 seconds: 40.77 ops/second llite.lustre-ffff93d507c93800.statahead_max=0 101 real 0m3.997s user 0m0.002s sys 0m0.358s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff93d507c93800.statahead_max=32 llite.lustre-ffff93d507c93800.statahead_max=32 101 real 0m1.737s user 0m0.003s sys 0m0.175s 'ls -l' 100 files with statahead: 1 sec statahead total: 105 statahead wrong: 2 agl total: 100 list_total: 30 fname_total: 63 hit_total: 4561 miss_total: 1284 swrong=2 ewrong=2 - open/close 607 (time 1784050789.20 total 10.00 last 60.68) total: 900 open/close in 13.17 seconds: 68.36 ops/second llite.lustre-ffff93d507c93800.statahead_max=0 1001 real 0m21.695s user 0m0.040s sys 0m3.018s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff93d507c93800.statahead_max=32 llite.lustre-ffff93d507c93800.statahead_max=32 1001 real 0m9.349s user 0m0.021s sys 0m1.479s 'ls -l' 1000 files with statahead: 9 sec statahead total: 106 statahead wrong: 2 agl total: 101 list_total: 31 fname_total: 63 hit_total: 5560 miss_total: 1285 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff93d507c93800.statahead_stats= statahead total: 106 statahead wrong: 2 agl total: 101 list_total: 31 fname_total: 63 hit_total: 5560 miss_total: 1285 mdc.lustre-MDT0000-mdc-ffff93d507c93800.batch_stats= snapshot_time: 1784050857.030468922 secs.nsecs start_time: 1784050760.604048092 secs.nsecs elapsed_time: 96.426420830 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 1 2 5 4: 1 2 7 8: 4 10 17 16: 18 46 64 32: 14 35 100 mdc.lustre-MDT0001-mdc-ffff93d507c93800.batch_stats= snapshot_time: 1784050857.030537970 secs.nsecs start_time: 1784050760.604184275 secs.nsecs elapsed_time: 96.426353695 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 3 7 7 8: 2 5 12 16: 23 58 71 32: 11 28 100 - open/close 878 (time 1784050870.05 total 10.01 last 87.75) - open/close 1870 (time 1784050880.05 total 20.01 last 99.18) - open/close 2959 (time 1784050890.05 total 30.01 last 108.87) - open/close 3993 (time 1784050900.06 total 40.01 last 103.36) - open/close 5034 (time 1784050910.06 total 50.02 last 104.07) - open/close 5614 (time 1784050920.07 total 60.03 last 57.94) - open/close 6376 (time 1784050930.08 total 70.04 last 76.11) - open/close 7374 (time 1784050940.09 total 80.05 last 99.74) - open/close 8291 (time 1784050950.09 total 90.05 last 91.63) - open/close 9224 (time 1784050960.10 total 100.06 last 93.19) - open/close 9971 (time 1784050970.11 total 110.07 last 74.67) total: 10000 open/close in 110.60 seconds: 90.41 ops/second llite.lustre-ffff93d507c93800.statahead_batch_max=0 llite.lustre-ffff93d507c93800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93d507c93800.stats=clear mdc.lustre-MDT0001-mdc-ffff93d507c93800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.101s user 0m0.316s sys 0m15.435s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93d507c93800.statahead_batch_max=32 llite.lustre-ffff93d507c93800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93d507c93800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff93d507c93800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93d507c93800.stats=clear mdc.lustre-MDT0001-mdc-ffff93d507c93800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m53.064s user 0m0.293s sys 0m14.905s batched statahead RPC= hit count=9977 mdc.lustre-MDT0000-mdc-ffff93d507c93800.batch_stats= snapshot_time: 1784051174.537323226 secs.nsecs start_time: 1784051047.450087993 secs.nsecs elapsed_time: 127.087235233 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff93d507c93800.batch_stats= snapshot_time: 1784051174.537399910 secs.nsecs start_time: 1784051047.450279412 secs.nsecs elapsed_time: 127.087120498 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff93d507c93800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93d507c93800.batch_stats=0 llite.lustre-ffff93d507c93800.statahead_max=2048 llite.lustre-ffff93d507c93800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff93d507c93800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93d507c93800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.06 seconds: 48.61 ops/second llite.lustre-ffff93d507c93800.statahead_max=0 101 real 0m1.894s user 0m0.002s sys 0m0.319s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff93d507c93800.statahead_max=2048 llite.lustre-ffff93d507c93800.statahead_max=2048 101 real 0m1.069s user 0m0.005s sys 0m0.190s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10076 miss_total: 24 swrong=0 ewrong=0 total: 900 open/close in 8.91 seconds: 101.01 ops/second llite.lustre-ffff93d507c93800.statahead_max=0 1001 real 0m20.473s user 0m0.034s sys 0m3.075s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff93d507c93800.statahead_max=2048 llite.lustre-ffff93d507c93800.statahead_max=2048 1001 real 0m6.721s user 0m0.023s sys 0m1.536s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11075 miss_total: 25 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff93d507c93800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11075 miss_total: 25 mdc.lustre-MDT0000-mdc-ffff93d507c93800.batch_stats= snapshot_time: 1784051510.298610301 secs.nsecs start_time: 1784051177.108777232 secs.nsecs elapsed_time: 333.189833069 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 22 22 8: 1 11 33 16: 1 11 44 32: 2 22 66 64: 1 11 77 128: 0 0 77 256: 2 22 100 mdc.lustre-MDT0001-mdc-ffff93d507c93800.batch_stats= snapshot_time: 1784051510.298677988 secs.nsecs start_time: 1784051177.108929498 secs.nsecs elapsed_time: 333.189748490 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 10 10 4: 0 0 10 8: 4 40 50 16: 1 10 60 32: 0 0 60 64: 2 20 80 128: 0 0 80 256: 2 20 100 - open/close 879 (time 1784051522.23 total 10.00 last 87.87) - open/close 1855 (time 1784051532.24 total 20.01 last 97.53) - open/close 2877 (time 1784051542.25 total 30.02 last 102.11) - open/close 3768 (time 1784051552.26 total 40.03 last 89.02) - open/close 4781 (time 1784051562.26 total 50.03 last 101.29) - open/close 5698 (time 1784051572.26 total 60.03 last 91.69) - open/close 6567 (time 1784051582.27 total 70.04 last 86.81) - open/close 7555 (time 1784051592.27 total 80.04 last 98.77) - open/close 8521 (time 1784051602.28 total 90.05 last 96.56) - open/close 9489 (time 1784051612.28 total 100.05 last 96.74) total: 10000 open/close in 107.14 seconds: 93.33 ops/second llite.lustre-ffff93d507c93800.statahead_batch_max=0 llite.lustre-ffff93d507c93800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93d507c93800.stats=clear mdc.lustre-MDT0001-mdc-ffff93d507c93800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.998s user 0m0.310s sys 0m16.489s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93d507c93800.statahead_batch_max=256 llite.lustre-ffff93d507c93800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93d507c93800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff93d507c93800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93d507c93800.stats=clear mdc.lustre-MDT0001-mdc-ffff93d507c93800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m5.683s user 0m0.313s sys 0m15.642s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff93d507c93800.batch_stats= snapshot_time: 1784051783.243837061 secs.nsecs start_time: 1784051697.243883676 secs.nsecs elapsed_time: 85.999953385 secs.nsecs subreqs per batch batches %% cum %% 1: 210 47 47 2: 76 17 64 4: 47 10 75 8: 31 6 81 16: 17 3 85 32: 23 5 90 64: 7 1 92 128: 1 0 92 256: 32 7 100 mdc.lustre-MDT0001-mdc-ffff93d507c93800.batch_stats= snapshot_time: 1784051783.243914221 secs.nsecs start_time: 1784051697.244024821 secs.nsecs elapsed_time: 85.999889400 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff93d507c93800.statahead_batch_max=64 llite.lustre-ffff93d507c93800.statahead_max=128