== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:02:25 (1785762145) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.batch_stats=0 llite.lustre-ffff9e1d831fa800.statahead_max=32 llite.lustre-ffff9e1d831fa800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.batch_stats=0 total: 100 open/close in 3.11 seconds: 32.18 ops/second llite.lustre-ffff9e1d831fa800.statahead_max=0 101 real 0m3.572s user 0m0.009s sys 0m0.333s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9e1d831fa800.statahead_max=32 llite.lustre-ffff9e1d831fa800.statahead_max=32 101 real 0m0.983s user 0m0.004s sys 0m0.156s 'ls -l' 100 files with statahead: 0 sec statahead total: 141 statahead wrong: 2 agl total: 134 list_total: 31 fname_total: 110 hit_total: 3870 miss_total: 1364 swrong=2 ewrong=2 - open/close 442 (time 1785762176.87 total 10.01 last 44.15) - open/close 778 (time 1785762186.88 total 20.02 last 33.57) total: 900 open/close in 21.33 seconds: 42.19 ops/second llite.lustre-ffff9e1d831fa800.statahead_max=0 1001 real 0m36.991s user 0m0.030s sys 0m3.005s 'ls -l' 1000 files without statahead: 37 sec llite.lustre-ffff9e1d831fa800.statahead_max=32 llite.lustre-ffff9e1d831fa800.statahead_max=32 1001 real 0m13.709s user 0m0.031s sys 0m1.477s 'ls -l' 1000 files with statahead: 14 sec statahead total: 142 statahead wrong: 2 agl total: 135 list_total: 32 fname_total: 110 hit_total: 4867 miss_total: 1367 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 38 seconds llite.lustre-ffff9e1d831fa800.statahead_stats= statahead total: 142 statahead wrong: 2 agl total: 135 list_total: 32 fname_total: 110 hit_total: 4867 miss_total: 1367 mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.batch_stats= snapshot_time: 1785762295.555533134 secs.nsecs start_time: 1785762150.678227382 secs.nsecs elapsed_time: 144.877305752 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 1 2 12 32: 34 87 100 - open/close 471 (time 1785762309.73 total 10.02 last 47.00) - open/close 957 (time 1785762319.75 total 20.04 last 48.49) - open/close 1472 (time 1785762329.76 total 30.05 last 51.46) - open/close 1964 (time 1785762339.77 total 40.06 last 49.14) - open/close 2274 (time 1785762349.77 total 50.07 last 30.99) - open/close 2675 (time 1785762359.81 total 60.11 last 39.93) - open/close 3064 (time 1785762369.82 total 70.12 last 38.87) - open/close 3645 (time 1785762379.83 total 80.12 last 58.08) - open/close 4200 (time 1785762389.84 total 90.13 last 55.42) - open/close 4763 (time 1785762399.85 total 100.15 last 56.24) - open/close 5227 (time 1785762409.87 total 110.16 last 46.31) - open/close 5786 (time 1785762419.88 total 120.17 last 55.86) - open/close 6280 (time 1785762429.89 total 130.18 last 49.34) - open/close 6764 (time 1785762439.90 total 140.19 last 48.36) - open/close 7373 (time 1785762449.92 total 150.21 last 60.78) - open/close 7877 (time 1785762459.93 total 160.22 last 50.34) - open/close 8551 (time 1785762469.93 total 170.22 last 67.40) - open/close 9081 (time 1785762479.94 total 180.24 last 52.91) - open/close 9610 (time 1785762489.95 total 190.25 last 52.85) total: 10000 open/close in 196.12 seconds: 50.99 ops/second llite.lustre-ffff9e1d831fa800.statahead_batch_max=0 llite.lustre-ffff9e1d831fa800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m31.500s user 0m0.239s sys 0m15.248s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9e1d831fa800.statahead_batch_max=32 llite.lustre-ffff9e1d831fa800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.stats=clear ls -l files in d123ad.sanity: 10001 real 2m12.306s user 0m0.246s sys 0m13.862s batched statahead RPC= hit count=9987 mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.batch_stats= snapshot_time: 1785762754.335720125 secs.nsecs start_time: 1785762602.520337856 secs.nsecs elapsed_time: 151.815382269 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.batch_stats=0 llite.lustre-ffff9e1d831fa800.statahead_max=2048 llite.lustre-ffff9e1d831fa800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.batch_stats=0 total: 100 open/close in 4.37 seconds: 22.88 ops/second llite.lustre-ffff9e1d831fa800.statahead_max=0 101 real 0m4.357s user 0m0.003s sys 0m0.338s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff9e1d831fa800.statahead_max=2048 llite.lustre-ffff9e1d831fa800.statahead_max=2048 101 real 0m1.271s user 0m0.003s sys 0m0.122s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10086 miss_total: 14 swrong=0 ewrong=0 - open/close 478 (time 1785763167.42 total 10.01 last 47.77) total: 900 open/close in 18.39 seconds: 48.93 ops/second llite.lustre-ffff9e1d831fa800.statahead_max=0 1001 real 0m38.267s user 0m0.029s sys 0m3.122s 'ls -l' 1000 files without statahead: 38 sec llite.lustre-ffff9e1d831fa800.statahead_max=2048 llite.lustre-ffff9e1d831fa800.statahead_max=2048 1001 real 0m10.005s user 0m0.019s sys 0m1.375s 'ls -l' 1000 files with statahead: 10 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 36 seconds llite.lustre-ffff9e1d831fa800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.batch_stats= snapshot_time: 1785763279.608857669 secs.nsecs start_time: 1785762757.827914254 secs.nsecs elapsed_time: 521.780943415 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 1 8 66 256: 4 33 100 - open/close 473 (time 1785763292.80 total 10.03 last 47.17) - open/close 963 (time 1785763302.81 total 20.03 last 48.98) - open/close 1535 (time 1785763312.81 total 30.04 last 57.17) - open/close 2224 (time 1785763322.82 total 40.04 last 68.86) - open/close 2785 (time 1785763332.82 total 50.04 last 56.10) - open/close 3304 (time 1785763342.82 total 60.04 last 51.90) - open/close 4011 (time 1785763352.82 total 70.05 last 70.69) - open/close 4893 (time 1785763362.83 total 80.05 last 88.14) - open/close 5510 (time 1785763372.84 total 90.06 last 61.64) - open/close 6047 (time 1785763382.86 total 100.08 last 53.58) - open/close 6595 (time 1785763392.87 total 110.09 last 54.75) - open/close 7067 (time 1785763402.87 total 120.10 last 47.16) - open/close 7499 (time 1785763412.92 total 130.14 last 43.01) - open/close 7839 (time 1785763422.92 total 140.14 last 34.00) - open/close 8288 (time 1785763432.94 total 150.16 last 44.82) - open/close 8630 (time 1785763442.94 total 160.17 last 34.19) - open/close 8932 (time 1785763452.95 total 170.18 last 30.17) - open/close 9363 (time 1785763462.97 total 180.20 last 43.02) - open/close 9880 (time 1785763472.97 total 190.20 last 51.69) total: 10000 open/close in 192.31 seconds: 52.00 ops/second llite.lustre-ffff9e1d831fa800.statahead_batch_max=0 llite.lustre-ffff9e1d831fa800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m32.056s user 0m0.279s sys 0m15.515s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9e1d831fa800.statahead_batch_max=256 llite.lustre-ffff9e1d831fa800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m20.668s user 0m0.281s sys 0m14.173s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9e1d831fa800.batch_stats= snapshot_time: 1785763680.931973448 secs.nsecs start_time: 1785763581.226637548 secs.nsecs elapsed_time: 99.705335900 secs.nsecs subreqs per batch batches %% cum %% 1: 207 49 49 2: 68 16 65 4: 53 12 78 8: 20 4 83 16: 21 5 88 32: 13 3 91 64: 3 0 91 128: 0 0 91 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9e1d831fa800.statahead_batch_max=64 llite.lustre-ffff9e1d831fa800.statahead_max=128