== sanity test 123ad: Verify batching statahead works correctly ========================================================== 19:49:43 (1787874583) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff95fe8648d800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff95fe8648d800.batch_stats=0 llite.lustre-ffff95fe8648d800.statahead_max=32 llite.lustre-ffff95fe8648d800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff95fe8648d800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff95fe8648d800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.81 seconds: 35.63 ops/second llite.lustre-ffff95fe8648d800.statahead_max=0 101 real 0m2.492s user 0m0.005s sys 0m0.309s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff95fe8648d800.statahead_max=32 llite.lustre-ffff95fe8648d800.statahead_max=32 101 real 0m1.087s user 0m0.003s sys 0m0.163s 'ls -l' 100 files with statahead: 1 sec statahead total: 167 statahead wrong: 2 agl total: 161 list_total: 31 fname_total: 119 hit_total: 3557 miss_total: 1012 swrong=2 ewrong=2 - open/close 839 (time 1787874611.20 total 10.00 last 83.90) total: 900 open/close in 10.98 seconds: 81.98 ops/second llite.lustre-ffff95fe8648d800.statahead_max=0 1001 real 0m19.991s user 0m0.036s sys 0m2.888s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff95fe8648d800.statahead_max=32 llite.lustre-ffff95fe8648d800.statahead_max=32 1001 real 0m8.278s user 0m0.028s sys 0m1.331s 'ls -l' 1000 files with statahead: 8 sec statahead total: 168 statahead wrong: 2 agl total: 162 list_total: 32 fname_total: 119 hit_total: 4553 miss_total: 1016 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff95fe8648d800.statahead_stats= statahead total: 168 statahead wrong: 2 agl total: 162 list_total: 32 fname_total: 119 hit_total: 4553 miss_total: 1016 mdc.lustre-MDT0000-mdc-ffff95fe8648d800.batch_stats= snapshot_time: 1787874672.592514177 secs.nsecs start_time: 1787874587.981040472 secs.nsecs elapsed_time: 84.611473705 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 1 2 5 8: 7 17 22 16: 18 45 67 32: 13 32 100 mdc.lustre-MDT0001-mdc-ffff95fe8648d800.batch_stats= snapshot_time: 1787874672.592582145 secs.nsecs start_time: 1787874587.981116757 secs.nsecs elapsed_time: 84.611465388 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 2 5 5 4: 2 5 10 8: 2 5 15 16: 20 51 66 32: 13 33 100 - open/close 818 (time 1787874684.83 total 10.00 last 81.76) - open/close 1538 (time 1787874694.83 total 20.01 last 71.95) - open/close 2450 (time 1787874704.83 total 30.01 last 91.20) - open/close 3297 (time 1787874714.86 total 40.03 last 84.52) - open/close 4015 (time 1787874724.86 total 50.04 last 71.75) - open/close 4768 (time 1787874734.87 total 60.04 last 75.29) - open/close 5388 (time 1787874744.87 total 70.04 last 61.99) - open/close 6138 (time 1787874754.87 total 80.05 last 74.97) - open/close 6918 (time 1787874764.87 total 90.05 last 77.98) - open/close 7929 (time 1787874774.88 total 100.05 last 101.07) - open/close 8996 (time 1787874784.88 total 110.06 last 106.65) - open/close 9982 (time 1787874794.89 total 120.07 last 98.51) total: 10000 open/close in 120.33 seconds: 83.11 ops/second llite.lustre-ffff95fe8648d800.statahead_batch_max=0 llite.lustre-ffff95fe8648d800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95fe8648d800.stats=clear mdc.lustre-MDT0001-mdc-ffff95fe8648d800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.804s user 0m0.344s sys 0m14.148s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95fe8648d800.statahead_batch_max=32 llite.lustre-ffff95fe8648d800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95fe8648d800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff95fe8648d800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95fe8648d800.stats=clear mdc.lustre-MDT0001-mdc-ffff95fe8648d800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m37.045s user 0m0.335s sys 0m13.236s batched statahead RPC= hit count=9984 mdc.lustre-MDT0000-mdc-ffff95fe8648d800.batch_stats= snapshot_time: 1787874981.857019689 secs.nsecs start_time: 1787874871.885373115 secs.nsecs elapsed_time: 109.971646574 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff95fe8648d800.batch_stats= snapshot_time: 1787874981.857210265 secs.nsecs start_time: 1787874871.885457375 secs.nsecs elapsed_time: 109.971752890 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff95fe8648d800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff95fe8648d800.batch_stats=0 llite.lustre-ffff95fe8648d800.statahead_max=2048 llite.lustre-ffff95fe8648d800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff95fe8648d800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff95fe8648d800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 1.79 seconds: 55.90 ops/second llite.lustre-ffff95fe8648d800.statahead_max=0 101 real 0m1.796s user 0m0.006s sys 0m0.318s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff95fe8648d800.statahead_max=2048 llite.lustre-ffff95fe8648d800.statahead_max=2048 101 real 0m0.702s user 0m0.004s sys 0m0.173s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10083 miss_total: 17 swrong=0 ewrong=0 total: 900 open/close in 8.42 seconds: 106.90 ops/second llite.lustre-ffff95fe8648d800.statahead_max=0 1001 real 0m27.333s user 0m0.030s sys 0m2.880s 'ls -l' 1000 files without statahead: 28 sec llite.lustre-ffff95fe8648d800.statahead_max=2048 llite.lustre-ffff95fe8648d800.statahead_max=2048 1001 real 0m8.363s user 0m0.040s sys 0m2.388s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff95fe8648d800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 mdc.lustre-MDT0000-mdc-ffff95fe8648d800.batch_stats= snapshot_time: 1787875319.184089994 secs.nsecs start_time: 1787874985.248366698 secs.nsecs elapsed_time: 333.935723296 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 10 10 4: 3 30 40 8: 0 0 40 16: 2 20 60 32: 2 20 80 64: 0 0 80 128: 0 0 80 256: 2 20 100 mdc.lustre-MDT0001-mdc-ffff95fe8648d800.batch_stats= snapshot_time: 1787875319.188374839 secs.nsecs start_time: 1787874985.248565854 secs.nsecs elapsed_time: 333.939808985 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 6 60 60 16: 2 20 80 32: 0 0 80 64: 0 0 80 128: 0 0 80 256: 2 20 100 - open/close 575 (time 1787875332.69 total 10.01 last 57.45) - open/close 1426 (time 1787875342.69 total 20.01 last 85.10) - open/close 2305 (time 1787875352.70 total 30.01 last 87.88) - open/close 3254 (time 1787875362.70 total 40.02 last 94.85) - open/close 4252 (time 1787875372.71 total 50.03 last 99.69) - open/close 5180 (time 1787875382.71 total 60.03 last 92.80) - open/close 6146 (time 1787875392.72 total 70.03 last 96.55) - open/close 7092 (time 1787875402.72 total 80.03 last 94.57) - open/close 8043 (time 1787875412.73 total 90.05 last 94.99) - open/close 8960 (time 1787875422.74 total 100.05 last 91.65) - open/close 9934 (time 1787875432.74 total 110.05 last 97.40) total: 10000 open/close in 110.67 seconds: 90.36 ops/second llite.lustre-ffff95fe8648d800.statahead_batch_max=0 llite.lustre-ffff95fe8648d800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95fe8648d800.stats=clear mdc.lustre-MDT0001-mdc-ffff95fe8648d800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.266s user 0m0.279s sys 0m13.476s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95fe8648d800.statahead_batch_max=256 llite.lustre-ffff95fe8648d800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95fe8648d800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff95fe8648d800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95fe8648d800.stats=clear mdc.lustre-MDT0001-mdc-ffff95fe8648d800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.147s user 0m0.317s sys 0m13.365s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff95fe8648d800.batch_stats= snapshot_time: 1787875558.305083651 secs.nsecs start_time: 1787875497.998472476 secs.nsecs elapsed_time: 60.306611175 secs.nsecs subreqs per batch batches %% cum %% 1: 128 41 41 2: 48 15 57 4: 36 11 68 8: 25 8 76 16: 12 3 80 32: 18 5 86 64: 7 2 88 128: 0 0 88 256: 34 11 100 mdc.lustre-MDT0001-mdc-ffff95fe8648d800.batch_stats= snapshot_time: 1787875558.305150280 secs.nsecs start_time: 1787875497.998613483 secs.nsecs elapsed_time: 60.306536797 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff95fe8648d800.statahead_batch_max=64 llite.lustre-ffff95fe8648d800.statahead_max=128