== sanity test 123ad: Verify batching statahead works correctly ========================================================== 21:42:15 (1786412535) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff91b387f01000.batch_stats=0 llite.lustre-ffff91b387f01000.statahead_max=32 llite.lustre-ffff91b387f01000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff91b387f01000.batch_stats=0 total: 100 open/close in 3.29 seconds: 30.37 ops/second llite.lustre-ffff91b387f01000.statahead_max=0 101 real 0m2.815s user 0m0.003s sys 0m0.314s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff91b387f01000.statahead_max=32 llite.lustre-ffff91b387f01000.statahead_max=32 101 real 0m1.214s user 0m0.005s sys 0m0.176s 'ls -l' 100 files with statahead: 2 sec statahead total: 101 statahead wrong: 2 agl total: 95 list_total: 30 fname_total: 71 hit_total: 3655 miss_total: 1651 swrong=2 ewrong=2 - open/close 705 (time 1786412565.10 total 10.00 last 70.49) total: 900 open/close in 13.00 seconds: 69.21 ops/second llite.lustre-ffff91b387f01000.statahead_max=0 1001 real 0m21.433s user 0m0.041s sys 0m3.112s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff91b387f01000.statahead_max=32 llite.lustre-ffff91b387f01000.statahead_max=32 1001 real 0m12.281s user 0m0.023s sys 0m1.485s 'ls -l' 1000 files with statahead: 12 sec statahead total: 102 statahead wrong: 2 agl total: 96 list_total: 31 fname_total: 71 hit_total: 4650 miss_total: 1656 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 31 seconds llite.lustre-ffff91b387f01000.statahead_stats= statahead total: 102 statahead wrong: 2 agl total: 96 list_total: 31 fname_total: 71 hit_total: 4650 miss_total: 1656 mdc.lustre-MDT0000-mdc-ffff91b387f01000.batch_stats= snapshot_time: 1786412647.319946060 secs.nsecs start_time: 1786412540.785386459 secs.nsecs elapsed_time: 106.534559601 secs.nsecs subreqs per batch batches %% cum %% 1: 2 4 4 2: 0 0 4 4: 0 0 4 8: 3 7 12 16: 2 4 17 32: 34 82 100 - open/close 522 (time 1786412660.39 total 10.00 last 52.18) - open/close 1051 (time 1786412670.40 total 20.01 last 52.88) - open/close 1656 (time 1786412680.41 total 30.02 last 60.45) - open/close 2232 (time 1786412690.44 total 40.04 last 57.44) - open/close 2911 (time 1786412700.44 total 50.05 last 67.89) - open/close 3568 (time 1786412710.45 total 60.05 last 65.65) - open/close 4136 (time 1786412720.46 total 70.07 last 56.73) - open/close 4607 (time 1786412730.49 total 80.09 last 46.98) - open/close 5211 (time 1786412740.49 total 90.10 last 60.35) - open/close 5915 (time 1786412750.50 total 100.11 last 70.35) - open/close 6427 (time 1786412760.51 total 110.12 last 51.16) - open/close 7008 (time 1786412770.51 total 120.12 last 58.06) - open/close 7710 (time 1786412780.53 total 130.13 last 70.12) - open/close 8250 (time 1786412790.54 total 140.15 last 53.93) - open/close 8788 (time 1786412800.56 total 150.16 last 53.70) - open/close 9376 (time 1786412810.56 total 160.17 last 58.79) total: 10000 open/close in 169.86 seconds: 58.87 ops/second llite.lustre-ffff91b387f01000.statahead_batch_max=0 llite.lustre-ffff91b387f01000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91b387f01000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.381s user 0m0.363s sys 0m15.690s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff91b387f01000.statahead_batch_max=32 llite.lustre-ffff91b387f01000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91b387f01000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff91b387f01000.stats=clear ls -l files in d123ad.sanity: 10001 real 2m1.537s user 0m0.362s sys 0m15.730s batched statahead RPC= hit count=9987 mdc.lustre-MDT0000-mdc-ffff91b387f01000.batch_stats= snapshot_time: 1786413042.015883726 secs.nsecs start_time: 1786412905.576427960 secs.nsecs elapsed_time: 136.439455766 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff91b387f01000.batch_stats=0 llite.lustre-ffff91b387f01000.statahead_max=2048 llite.lustre-ffff91b387f01000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff91b387f01000.batch_stats=0 total: 100 open/close in 2.42 seconds: 41.32 ops/second llite.lustre-ffff91b387f01000.statahead_max=0 101 real 0m2.525s user 0m0.005s sys 0m0.376s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff91b387f01000.statahead_max=2048 llite.lustre-ffff91b387f01000.statahead_max=2048 101 real 0m1.085s user 0m0.005s sys 0m0.190s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10086 miss_total: 14 swrong=0 ewrong=0 - open/close 578 (time 1786413435.95 total 10.00 last 57.78) total: 900 open/close in 15.54 seconds: 57.92 ops/second llite.lustre-ffff91b387f01000.statahead_max=0 1001 real 0m29.957s user 0m0.038s sys 0m3.527s 'ls -l' 1000 files without statahead: 30 sec llite.lustre-ffff91b387f01000.statahead_max=2048 llite.lustre-ffff91b387f01000.statahead_max=2048 1001 real 0m10.647s user 0m0.029s sys 0m1.617s 'ls -l' 1000 files with statahead: 11 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff91b387f01000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 mdc.lustre-MDT0000-mdc-ffff91b387f01000.batch_stats= snapshot_time: 1786413521.361379444 secs.nsecs start_time: 1786413044.728232636 secs.nsecs elapsed_time: 476.633146808 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 617 (time 1786413533.79 total 10.01 last 61.63) - open/close 1284 (time 1786413543.79 total 20.01 last 66.68) - open/close 1910 (time 1786413553.79 total 30.02 last 62.59) - open/close 2632 (time 1786413563.80 total 40.02 last 72.14) - open/close 3152 (time 1786413573.81 total 50.04 last 51.93) - open/close 3812 (time 1786413583.82 total 60.05 last 65.93) - open/close 4366 (time 1786413593.83 total 70.05 last 55.38) - open/close 5023 (time 1786413603.83 total 80.05 last 65.70) - open/close 5771 (time 1786413613.84 total 90.06 last 74.72) - open/close 6420 (time 1786413623.86 total 100.08 last 64.74) - open/close 6862 (time 1786413633.87 total 110.09 last 44.16) - open/close 7347 (time 1786413643.88 total 120.10 last 48.47) - open/close 7911 (time 1786413653.91 total 130.13 last 56.22) - open/close 8338 (time 1786413663.94 total 140.16 last 42.59) - open/close 8912 (time 1786413673.95 total 150.17 last 57.35) - open/close 9604 (time 1786413683.96 total 160.18 last 69.13) total: 10000 open/close in 165.93 seconds: 60.27 ops/second llite.lustre-ffff91b387f01000.statahead_batch_max=0 llite.lustre-ffff91b387f01000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91b387f01000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.067s user 0m0.303s sys 0m15.349s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff91b387f01000.statahead_batch_max=256 llite.lustre-ffff91b387f01000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91b387f01000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff91b387f01000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.240s user 0m0.297s sys 0m15.125s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff91b387f01000.batch_stats= snapshot_time: 1786413843.898523452 secs.nsecs start_time: 1786413768.285575690 secs.nsecs elapsed_time: 75.612947762 secs.nsecs subreqs per batch batches %% cum %% 1: 248 54 54 2: 72 15 70 4: 38 8 78 8: 27 5 84 16: 18 3 88 32: 14 3 91 64: 6 1 92 128: 1 0 92 256: 33 7 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff91b387f01000.statahead_batch_max=64 llite.lustre-ffff91b387f01000.statahead_max=128