== sanity test 123ad: Verify batching statahead works correctly ========================================================== 18:33:56 (1786833236) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.batch_stats=0 llite.lustre-ffff8c9348cd8800.statahead_max=32 llite.lustre-ffff8c9348cd8800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.batch_stats=0 total: 100 open/close in 3.48 seconds: 28.74 ops/second llite.lustre-ffff8c9348cd8800.statahead_max=0 101 real 0m2.626s user 0m0.007s sys 0m0.393s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8c9348cd8800.statahead_max=32 llite.lustre-ffff8c9348cd8800.statahead_max=32 101 real 0m1.131s user 0m0.008s sys 0m0.185s 'ls -l' 100 files with statahead: 1 sec statahead total: 95 statahead wrong: 2 agl total: 89 list_total: 30 fname_total: 64 hit_total: 3705 miss_total: 1578 swrong=2 ewrong=2 - open/close 571 (time 1786833266.31 total 10.00 last 57.09) total: 900 open/close in 15.94 seconds: 56.46 ops/second llite.lustre-ffff8c9348cd8800.statahead_max=0 1001 real 0m26.147s user 0m0.043s sys 0m3.473s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff8c9348cd8800.statahead_max=32 llite.lustre-ffff8c9348cd8800.statahead_max=32 1001 real 0m11.434s user 0m0.039s sys 0m1.640s 'ls -l' 1000 files with statahead: 12 sec statahead total: 96 statahead wrong: 2 agl total: 90 list_total: 31 fname_total: 64 hit_total: 4702 miss_total: 1581 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff8c9348cd8800.statahead_stats= statahead total: 96 statahead wrong: 2 agl total: 90 list_total: 31 fname_total: 64 hit_total: 4702 miss_total: 1581 mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.batch_stats= snapshot_time: 1786833349.504489691 secs.nsecs start_time: 1786833241.816236798 secs.nsecs elapsed_time: 107.688252893 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 625 (time 1786833362.39 total 10.01 last 62.41) - open/close 1225 (time 1786833372.40 total 20.03 last 59.90) - open/close 1909 (time 1786833382.41 total 30.04 last 68.34) - open/close 2436 (time 1786833392.44 total 40.07 last 52.54) - open/close 3142 (time 1786833402.45 total 50.07 last 70.58) - open/close 3792 (time 1786833412.46 total 60.08 last 64.95) - open/close 4443 (time 1786833422.46 total 70.09 last 65.05) - open/close 5083 (time 1786833432.47 total 80.10 last 63.93) - open/close 5754 (time 1786833442.47 total 90.10 last 67.10) - open/close 6409 (time 1786833452.48 total 100.11 last 65.45) - open/close 7035 (time 1786833462.48 total 110.11 last 62.58) - open/close 7680 (time 1786833472.49 total 120.11 last 64.48) - open/close 8285 (time 1786833482.49 total 130.12 last 60.46) - open/close 8956 (time 1786833492.50 total 140.12 last 67.09) - open/close 9567 (time 1786833502.53 total 150.15 last 60.92) total: 10000 open/close in 157.15 seconds: 63.64 ops/second llite.lustre-ffff8c9348cd8800.statahead_batch_max=0 llite.lustre-ffff8c9348cd8800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.045s user 0m0.334s sys 0m17.127s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8c9348cd8800.statahead_batch_max=32 llite.lustre-ffff8c9348cd8800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m49.273s user 0m0.314s sys 0m15.374s batched statahead RPC= hit count=9988 mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.batch_stats= snapshot_time: 1786833696.469361190 secs.nsecs start_time: 1786833575.026551644 secs.nsecs elapsed_time: 121.442809546 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.batch_stats=0 llite.lustre-ffff8c9348cd8800.statahead_max=2048 llite.lustre-ffff8c9348cd8800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.batch_stats=0 total: 100 open/close in 3.04 seconds: 32.89 ops/second llite.lustre-ffff8c9348cd8800.statahead_max=0 101 real 0m2.649s user 0m0.005s sys 0m0.336s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8c9348cd8800.statahead_max=2048 llite.lustre-ffff8c9348cd8800.statahead_max=2048 101 real 0m0.962s user 0m0.004s sys 0m0.177s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10087 miss_total: 13 swrong=0 ewrong=0 - open/close 603 (time 1786834049.20 total 10.01 last 60.23) total: 900 open/close in 15.62 seconds: 57.60 ops/second llite.lustre-ffff8c9348cd8800.statahead_max=0 1001 real 0m23.987s user 0m0.036s sys 0m3.185s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff8c9348cd8800.statahead_max=2048 llite.lustre-ffff8c9348cd8800.statahead_max=2048 1001 real 0m9.063s user 0m0.023s sys 0m1.528s 'ls -l' 1000 files with statahead: 9 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff8c9348cd8800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.batch_stats= snapshot_time: 1786834127.773937858 secs.nsecs start_time: 1786833698.991679536 secs.nsecs elapsed_time: 428.782258322 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 620 (time 1786834140.66 total 10.02 last 61.85) - open/close 1250 (time 1786834150.66 total 20.03 last 62.97) - open/close 1946 (time 1786834160.67 total 30.04 last 69.52) - open/close 2615 (time 1786834170.67 total 40.04 last 66.89) - open/close 3164 (time 1786834180.68 total 50.05 last 54.87) - open/close 3779 (time 1786834190.68 total 60.05 last 61.50) - open/close 4444 (time 1786834200.69 total 70.05 last 66.46) - open/close 4988 (time 1786834210.69 total 80.06 last 54.35) - open/close 5554 (time 1786834220.70 total 90.07 last 56.58) - open/close 6081 (time 1786834230.70 total 100.07 last 52.69) - open/close 6638 (time 1786834240.71 total 110.08 last 55.62) - open/close 7241 (time 1786834250.73 total 120.09 last 60.23) - open/close 7909 (time 1786834260.73 total 130.10 last 66.78) - open/close 8500 (time 1786834270.74 total 140.11 last 59.03) - open/close 9105 (time 1786834280.76 total 150.12 last 60.41) - open/close 9732 (time 1786834290.76 total 160.13 last 62.68) total: 10000 open/close in 164.00 seconds: 60.98 ops/second llite.lustre-ffff8c9348cd8800.statahead_batch_max=0 llite.lustre-ffff8c9348cd8800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.353s user 0m0.298s sys 0m16.347s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8c9348cd8800.statahead_batch_max=256 llite.lustre-ffff8c9348cd8800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.054s user 0m0.309s sys 0m17.543s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8c9348cd8800.batch_stats= snapshot_time: 1786834435.147357817 secs.nsecs start_time: 1786834365.688060790 secs.nsecs elapsed_time: 69.459297027 secs.nsecs subreqs per batch batches %% cum %% 1: 217 54 54 2: 58 14 69 4: 28 7 76 8: 19 4 80 16: 22 5 86 32: 16 4 90 64: 3 0 91 128: 1 0 91 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 1 seconds: 0.000000 unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8c9348cd8800.statahead_batch_max=64 llite.lustre-ffff8c9348cd8800.statahead_max=128