== sanity test 123ad: Verify batching statahead works correctly ========================================================== 11:10:05 (1785942605) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff987a05216000.batch_stats=0 llite.lustre-ffff987a05216000.statahead_max=32 llite.lustre-ffff987a05216000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff987a05216000.batch_stats=0 total: 100 open/close in 2.51 seconds: 39.79 ops/second llite.lustre-ffff987a05216000.statahead_max=0 101 real 0m2.548s user 0m0.004s sys 0m0.279s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff987a05216000.statahead_max=32 llite.lustre-ffff987a05216000.statahead_max=32 101 real 0m0.869s user 0m0.001s sys 0m0.143s 'ls -l' 100 files with statahead: 1 sec statahead total: 154 statahead wrong: 2 agl total: 147 list_total: 30 fname_total: 121 hit_total: 3544 miss_total: 1490 swrong=2 ewrong=2 - open/close 610 (time 1785942634.01 total 10.01 last 60.93) total: 900 open/close in 15.70 seconds: 57.33 ops/second llite.lustre-ffff987a05216000.statahead_max=0 1001 real 0m35.901s user 0m0.045s sys 0m2.748s 'ls -l' 1000 files without statahead: 36 sec llite.lustre-ffff987a05216000.statahead_max=32 llite.lustre-ffff987a05216000.statahead_max=32 1001 real 0m15.437s user 0m0.025s sys 0m1.312s 'ls -l' 1000 files with statahead: 15 sec statahead total: 155 statahead wrong: 2 agl total: 148 list_total: 31 fname_total: 121 hit_total: 4539 miss_total: 1495 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 31 seconds llite.lustre-ffff987a05216000.statahead_stats= statahead total: 155 statahead wrong: 2 agl total: 148 list_total: 31 fname_total: 121 hit_total: 4539 miss_total: 1495 mdc.lustre-MDT0000-mdc-ffff987a05216000.batch_stats= snapshot_time: 1785942742.032234509 secs.nsecs start_time: 1785942611.011796697 secs.nsecs elapsed_time: 131.020437812 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 446 (time 1785942756.06 total 10.02 last 44.52) - open/close 1192 (time 1785942766.07 total 20.03 last 74.52) - open/close 1692 (time 1785942776.09 total 30.05 last 49.90) - open/close 2143 (time 1785942786.12 total 40.07 last 44.99) - open/close 2678 (time 1785942796.12 total 50.08 last 53.48) - open/close 3166 (time 1785942806.12 total 60.08 last 48.79) - open/close 3746 (time 1785942816.13 total 70.09 last 57.95) - open/close 4435 (time 1785942826.14 total 80.09 last 68.87) - open/close 4986 (time 1785942836.15 total 90.11 last 55.03) - open/close 5436 (time 1785942846.18 total 100.14 last 44.86) - open/close 5956 (time 1785942856.19 total 110.15 last 51.94) - open/close 6539 (time 1785942866.20 total 120.16 last 58.23) - open/close 7161 (time 1785942876.25 total 130.21 last 61.91) - open/close 7722 (time 1785942886.26 total 140.22 last 56.02) - open/close 8176 (time 1785942896.28 total 150.23 last 45.35) - open/close 8741 (time 1785942906.29 total 160.24 last 56.43) - open/close 9374 (time 1785942916.30 total 170.25 last 63.25) total: 10000 open/close in 179.80 seconds: 55.62 ops/second llite.lustre-ffff987a05216000.statahead_batch_max=0 llite.lustre-ffff987a05216000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff987a05216000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m29.319s user 0m0.304s sys 0m13.819s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff987a05216000.statahead_batch_max=32 llite.lustre-ffff987a05216000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff987a05216000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff987a05216000.stats=clear ls -l files in d123ad.sanity: 10001 real 2m19.581s user 0m0.244s sys 0m12.349s batched statahead RPC= hit count=9971 mdc.lustre-MDT0000-mdc-ffff987a05216000.batch_stats= snapshot_time: 1785943183.089306961 secs.nsecs start_time: 1785943027.528061464 secs.nsecs elapsed_time: 155.561245497 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 0 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff987a05216000.batch_stats=0 llite.lustre-ffff987a05216000.statahead_max=2048 llite.lustre-ffff987a05216000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff987a05216000.batch_stats=0 total: 100 open/close in 3.05 seconds: 32.76 ops/second llite.lustre-ffff987a05216000.statahead_max=0 101 real 0m3.346s user 0m0.003s sys 0m0.278s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff987a05216000.statahead_max=2048 llite.lustre-ffff987a05216000.statahead_max=2048 101 real 0m1.055s user 0m0.005s sys 0m0.164s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10070 miss_total: 30 swrong=0 ewrong=0 - open/close 634 (time 1785943555.45 total 10.02 last 63.26) total: 900 open/close in 14.45 seconds: 62.28 ops/second llite.lustre-ffff987a05216000.statahead_max=0 1001 real 0m30.675s user 0m0.038s sys 0m2.729s 'ls -l' 1000 files without statahead: 31 sec llite.lustre-ffff987a05216000.statahead_max=2048 llite.lustre-ffff987a05216000.statahead_max=2048 1001 real 0m7.631s user 0m0.024s sys 0m1.233s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11069 miss_total: 31 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 31 seconds llite.lustre-ffff987a05216000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11069 miss_total: 31 mdc.lustre-MDT0000-mdc-ffff987a05216000.batch_stats= snapshot_time: 1785943645.123686050 secs.nsecs start_time: 1785943186.213161731 secs.nsecs elapsed_time: 458.910524319 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 537 (time 1785943658.99 total 10.01 last 53.63) - open/close 922 (time 1785943669.02 total 20.04 last 38.40) - open/close 1347 (time 1785943679.03 total 30.05 last 42.46) - open/close 1900 (time 1785943689.03 total 40.06 last 55.25) - open/close 2360 (time 1785943699.04 total 50.06 last 45.97) - open/close 2862 (time 1785943709.04 total 60.07 last 50.19) - open/close 3304 (time 1785943719.08 total 70.10 last 44.04) - open/close 3795 (time 1785943729.10 total 80.12 last 49.00) - open/close 4264 (time 1785943739.10 total 90.13 last 46.88) - open/close 4715 (time 1785943749.11 total 100.14 last 45.05) - open/close 5347 (time 1785943759.12 total 110.14 last 63.17) - open/close 6258 (time 1785943769.12 total 120.15 last 91.06) - open/close 6803 (time 1785943779.13 total 130.15 last 54.49) - open/close 7634 (time 1785943789.15 total 140.17 last 82.93) - open/close 8159 (time 1785943799.15 total 150.17 last 52.47) - open/close 8659 (time 1785943809.18 total 160.20 last 49.86) - open/close 9127 (time 1785943819.22 total 170.25 last 46.60) - open/close 9805 (time 1785943829.23 total 180.26 last 67.72) total: 10000 open/close in 183.98 seconds: 54.35 ops/second llite.lustre-ffff987a05216000.statahead_batch_max=0 llite.lustre-ffff987a05216000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff987a05216000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m23.423s user 0m0.287s sys 0m13.011s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff987a05216000.statahead_batch_max=256 llite.lustre-ffff987a05216000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff987a05216000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff987a05216000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.560s user 0m0.259s sys 0m12.351s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff987a05216000.batch_stats= snapshot_time: 1785944009.445187910 secs.nsecs start_time: 1785943933.721215513 secs.nsecs elapsed_time: 75.723972397 secs.nsecs subreqs per batch batches %% cum %% 1: 138 42 42 2: 67 20 63 4: 29 8 72 8: 24 7 79 16: 10 3 82 32: 14 4 87 64: 6 1 89 128: 1 0 89 256: 34 10 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff987a05216000.statahead_batch_max=64 llite.lustre-ffff987a05216000.statahead_max=128