== sanity test 123ad: Verify batching statahead works correctly ========================================================== 14:58:34 (1777575514) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.batch_stats=0 llite.lustre-ffff8de2e0f5c000.statahead_max=32 llite.lustre-ffff8de2e0f5c000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.batch_stats=0 total: 100 open/close in 2.34 seconds: 42.78 ops/second llite.lustre-ffff8de2e0f5c000.statahead_max=0 101 real 0m2.196s user 0m0.005s sys 0m0.282s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8de2e0f5c000.statahead_max=32 llite.lustre-ffff8de2e0f5c000.statahead_max=32 101 real 0m1.145s user 0m0.006s sys 0m0.165s 'ls -l' 100 files with statahead: 1 sec statahead total: 94 statahead wrong: 1 agl total: 89 list_total: 30 fname_total: 64 hit_total: 14406 miss_total: 250 swrong=1 ewrong=1 - open/close 636 (time 1777575540.44 total 10.01 last 63.54) total: 900 open/close in 14.99 seconds: 60.04 ops/second llite.lustre-ffff8de2e0f5c000.statahead_max=0 1001 real 0m26.041s user 0m0.038s sys 0m3.105s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff8de2e0f5c000.statahead_max=32 llite.lustre-ffff8de2e0f5c000.statahead_max=32 1001 real 0m12.089s user 0m0.037s sys 0m1.484s 'ls -l' 1000 files with statahead: 12 sec statahead total: 95 statahead wrong: 1 agl total: 90 list_total: 31 fname_total: 64 hit_total: 15403 miss_total: 253 swrong=1 ewrong=1 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff8de2e0f5c000.statahead_stats= statahead total: 95 statahead wrong: 1 agl total: 90 list_total: 31 fname_total: 64 hit_total: 15403 miss_total: 253 mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.batch_stats= snapshot_time: 1777575626.257951441 secs.nsecs start_time: 1777575518.493455501 secs.nsecs elapsed_time: 107.764495940 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 605 (time 1777575639.35 total 10.00 last 60.49) - open/close 1365 (time 1777575649.36 total 20.01 last 75.93) - open/close 2003 (time 1777575659.36 total 30.02 last 63.77) - open/close 2672 (time 1777575669.37 total 40.02 last 66.86) - open/close 3420 (time 1777575679.37 total 50.03 last 74.78) - open/close 4071 (time 1777575689.38 total 60.03 last 65.08) - open/close 4911 (time 1777575699.38 total 70.03 last 84.00) - open/close 5736 (time 1777575709.38 total 80.03 last 82.50) - open/close 6515 (time 1777575719.38 total 90.03 last 77.88) - open/close 7275 (time 1777575729.40 total 100.05 last 75.88) - open/close 8043 (time 1777575739.41 total 110.06 last 76.70) - open/close 8783 (time 1777575749.41 total 120.06 last 73.98) - open/close 9550 (time 1777575759.44 total 130.09 last 76.49) total: 10000 open/close in 136.38 seconds: 73.32 ops/second llite.lustre-ffff8de2e0f5c000.statahead_batch_max=0 llite.lustre-ffff8de2e0f5c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.250s user 0m0.337s sys 0m14.551s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8de2e0f5c000.statahead_batch_max=32 llite.lustre-ffff8de2e0f5c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m35.586s user 0m0.321s sys 0m14.103s batched statahead RPC= hit count=9987 mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.batch_stats= snapshot_time: 1777575942.263511718 secs.nsecs start_time: 1777575831.138121135 secs.nsecs elapsed_time: 111.125390583 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.batch_stats=0 llite.lustre-ffff8de2e0f5c000.statahead_max=2048 llite.lustre-ffff8de2e0f5c000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.batch_stats=0 total: 100 open/close in 2.11 seconds: 47.49 ops/second llite.lustre-ffff8de2e0f5c000.statahead_max=0 101 real 0m2.027s user 0m0.006s sys 0m0.309s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8de2e0f5c000.statahead_max=2048 llite.lustre-ffff8de2e0f5c000.statahead_max=2048 101 real 0m0.799s user 0m0.006s sys 0m0.203s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10086 miss_total: 14 swrong=0 ewrong=0 - open/close 817 (time 1777576231.13 total 10.02 last 81.51) total: 900 open/close in 11.14 seconds: 80.78 ops/second llite.lustre-ffff8de2e0f5c000.statahead_max=0 1001 real 0m19.943s user 0m0.030s sys 0m3.065s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff8de2e0f5c000.statahead_max=2048 llite.lustre-ffff8de2e0f5c000.statahead_max=2048 1001 real 0m5.662s user 0m0.025s sys 0m1.414s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 15 seconds llite.lustre-ffff8de2e0f5c000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.batch_stats= snapshot_time: 1777576285.333581220 secs.nsecs start_time: 1777575945.016141612 secs.nsecs elapsed_time: 340.317439608 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 0 0 40 64: 1 10 50 128: 1 10 60 256: 4 40 100 - open/close 909 (time 1777576297.40 total 10.01 last 90.78) - open/close 1721 (time 1777576307.42 total 20.03 last 81.08) - open/close 2532 (time 1777576317.42 total 30.03 last 81.09) - open/close 3300 (time 1777576327.42 total 40.03 last 76.78) - open/close 4139 (time 1777576337.42 total 50.03 last 83.90) - open/close 4990 (time 1777576347.43 total 60.04 last 85.03) - open/close 5798 (time 1777576357.45 total 70.06 last 80.60) - open/close 6518 (time 1777576367.46 total 80.07 last 71.96) - open/close 7194 (time 1777576377.47 total 90.08 last 67.50) - open/close 7985 (time 1777576387.48 total 100.09 last 79.01) - open/close 8818 (time 1777576397.49 total 110.10 last 83.25) - open/close 9558 (time 1777576407.49 total 120.10 last 73.97) total: 10000 open/close in 125.98 seconds: 79.38 ops/second llite.lustre-ffff8de2e0f5c000.statahead_batch_max=0 llite.lustre-ffff8de2e0f5c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m46.807s user 0m0.281s sys 0m14.873s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8de2e0f5c000.statahead_batch_max=256 llite.lustre-ffff8de2e0f5c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m47.367s user 0m0.268s sys 0m14.963s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8de2e0f5c000.batch_stats= snapshot_time: 1777576527.269977265 secs.nsecs start_time: 1777576468.511585303 secs.nsecs elapsed_time: 58.758391962 secs.nsecs subreqs per batch batches % cum % 1: 247 49 49 2: 75 15 64 4: 76 15 80 8: 30 6 86 16: 19 3 90 32: 10 2 92 64: 5 1 93 128: 1 0 93 256: 33 6 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8de2e0f5c000.statahead_batch_max=64 llite.lustre-ffff8de2e0f5c000.statahead_max=128