== sanity test 123ad: Verify batching statahead works correctly ========================================================== 14:10:17 (1786039817) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.batch_stats=0 llite.lustre-ffff9cb5d009a000.statahead_max=32 llite.lustre-ffff9cb5d009a000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.batch_stats=0 total: 100 open/close in 1.91 seconds: 52.40 ops/second llite.lustre-ffff9cb5d009a000.statahead_max=0 101 real 0m4.293s user 0m0.004s sys 0m0.350s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff9cb5d009a000.statahead_max=32 llite.lustre-ffff9cb5d009a000.statahead_max=32 101 real 0m1.463s user 0m0.000s sys 0m0.165s 'ls -l' 100 files with statahead: 1 sec statahead total: 169 statahead wrong: 2 agl total: 162 list_total: 31 fname_total: 128 hit_total: 3903 miss_total: 1647 swrong=2 ewrong=2 - open/close 864 (time 1786039848.29 total 10.00 last 86.38) total: 900 open/close in 10.26 seconds: 87.76 ops/second llite.lustre-ffff9cb5d009a000.statahead_max=0 1001 real 0m29.672s user 0m0.030s sys 0m3.154s 'ls -l' 1000 files without statahead: 29 sec llite.lustre-ffff9cb5d009a000.statahead_max=32 llite.lustre-ffff9cb5d009a000.statahead_max=32 1001 real 0m13.533s user 0m0.022s sys 0m1.451s 'ls -l' 1000 files with statahead: 14 sec statahead total: 170 statahead wrong: 2 agl total: 163 list_total: 32 fname_total: 128 hit_total: 4899 miss_total: 1651 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff9cb5d009a000.statahead_stats= statahead total: 170 statahead wrong: 2 agl total: 163 list_total: 32 fname_total: 128 hit_total: 4899 miss_total: 1651 mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.batch_stats= snapshot_time: 1786039933.352737213 secs.nsecs start_time: 1786039823.095181229 secs.nsecs elapsed_time: 110.257555984 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 547 (time 1786039946.83 total 10.01 last 54.65) - open/close 1089 (time 1786039956.85 total 20.03 last 54.08) - open/close 1759 (time 1786039966.87 total 30.05 last 66.91) - open/close 2243 (time 1786039976.87 total 40.05 last 48.36) - open/close 2820 (time 1786039986.89 total 50.07 last 57.61) - open/close 3421 (time 1786039996.91 total 60.09 last 59.99) - open/close 3998 (time 1786040006.91 total 70.09 last 57.70) - open/close 4614 (time 1786040016.91 total 80.09 last 61.59) - open/close 5185 (time 1786040026.91 total 90.09 last 57.10) - open/close 5619 (time 1786040036.93 total 100.11 last 43.32) - open/close 5986 (time 1786040046.96 total 110.14 last 36.59) - open/close 6351 (time 1786040056.98 total 120.16 last 36.43) - open/close 6810 (time 1786040066.99 total 130.17 last 45.85) - open/close 7478 (time 1786040077.00 total 140.18 last 66.70) - open/close 8226 (time 1786040087.02 total 150.20 last 74.70) - open/close 8911 (time 1786040097.02 total 160.20 last 68.46) - open/close 9757 (time 1786040107.03 total 170.21 last 84.56) total: 10000 open/close in 174.68 seconds: 57.25 ops/second llite.lustre-ffff9cb5d009a000.statahead_batch_max=0 llite.lustre-ffff9cb5d009a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m22.272s user 0m0.290s sys 0m14.443s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9cb5d009a000.statahead_batch_max=32 llite.lustre-ffff9cb5d009a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.stats=clear ls -l files in d123ad.sanity: 10001 real 2m24.963s user 0m0.272s sys 0m14.175s batched statahead RPC= hit count=9984 mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.batch_stats= snapshot_time: 1786040382.457532479 secs.nsecs start_time: 1786040207.995953795 secs.nsecs elapsed_time: 174.461578684 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 0 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.batch_stats=0 llite.lustre-ffff9cb5d009a000.statahead_max=2048 llite.lustre-ffff9cb5d009a000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.batch_stats=0 total: 100 open/close in 4.50 seconds: 22.25 ops/second llite.lustre-ffff9cb5d009a000.statahead_max=0 101 real 0m3.571s user 0m0.004s sys 0m0.326s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff9cb5d009a000.statahead_max=2048 llite.lustre-ffff9cb5d009a000.statahead_max=2048 101 real 0m1.902s user 0m0.001s sys 0m0.192s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10083 miss_total: 17 swrong=0 ewrong=0 - open/close 634 (time 1786040743.31 total 10.01 last 63.36) total: 900 open/close in 13.28 seconds: 67.76 ops/second llite.lustre-ffff9cb5d009a000.statahead_max=0 1001 real 0m33.492s user 0m0.031s sys 0m3.120s 'ls -l' 1000 files without statahead: 33 sec llite.lustre-ffff9cb5d009a000.statahead_max=2048 llite.lustre-ffff9cb5d009a000.statahead_max=2048 1001 real 0m12.730s user 0m0.031s sys 0m1.440s 'ls -l' 1000 files with statahead: 13 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff9cb5d009a000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.batch_stats= snapshot_time: 1786040834.011127015 secs.nsecs start_time: 1786040385.616844138 secs.nsecs elapsed_time: 448.394282877 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 837 (time 1786040847.33 total 10.02 last 83.53) - open/close 1321 (time 1786040857.36 total 20.05 last 48.28) - open/close 1774 (time 1786040867.37 total 30.06 last 45.24) - open/close 2327 (time 1786040877.38 total 40.07 last 55.27) - open/close 2856 (time 1786040887.38 total 50.07 last 52.89) - open/close 3391 (time 1786040897.39 total 60.07 last 53.47) - open/close 3883 (time 1786040907.40 total 70.09 last 49.11) - open/close 4243 (time 1786040917.44 total 80.12 last 35.87) - open/close 4728 (time 1786040927.45 total 90.14 last 48.45) - open/close 5279 (time 1786040937.45 total 100.14 last 55.09) - open/close 6127 (time 1786040947.46 total 110.14 last 84.76) - open/close 6754 (time 1786040957.46 total 120.15 last 62.67) - open/close 7236 (time 1786040967.47 total 130.15 last 48.18) - open/close 7650 (time 1786040977.50 total 140.19 last 41.26) - open/close 8095 (time 1786040987.50 total 150.19 last 44.48) - open/close 8930 (time 1786040997.50 total 160.19 last 83.50) - open/close 9527 (time 1786041007.52 total 170.20 last 59.62) total: 10000 open/close in 177.92 seconds: 56.20 ops/second llite.lustre-ffff9cb5d009a000.statahead_batch_max=0 llite.lustre-ffff9cb5d009a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m38.017s user 0m0.312s sys 0m15.832s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9cb5d009a000.statahead_batch_max=256 llite.lustre-ffff9cb5d009a000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m12.360s user 0m0.248s sys 0m14.181s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9cb5d009a000.batch_stats= snapshot_time: 1786041226.836724958 secs.nsecs start_time: 1786041129.701467072 secs.nsecs elapsed_time: 97.135257886 secs.nsecs subreqs per batch batches %% cum %% 1: 305 57 57 2: 76 14 72 4: 40 7 80 8: 31 5 85 16: 19 3 89 32: 20 3 93 64: 1 0 93 128: 1 0 93 256: 33 6 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9cb5d009a000.statahead_batch_max=64 llite.lustre-ffff9cb5d009a000.statahead_max=128