== sanity test 123ad: Verify batching statahead works correctly ========================================================== 03:31:39 (1785396699) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9d1358449800.batch_stats=0 llite.lustre-ffff9d1358449800.statahead_max=32 llite.lustre-ffff9d1358449800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9d1358449800.batch_stats=0 total: 100 open/close in 4.12 seconds: 24.25 ops/second llite.lustre-ffff9d1358449800.statahead_max=0 101 real 0m4.010s user 0m0.002s sys 0m0.352s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff9d1358449800.statahead_max=32 llite.lustre-ffff9d1358449800.statahead_max=32 101 real 0m1.521s user 0m0.004s sys 0m0.173s 'ls -l' 100 files with statahead: 1 sec statahead total: 150 statahead wrong: 2 agl total: 142 list_total: 30 fname_total: 104 hit_total: 3756 miss_total: 1690 swrong=2 ewrong=2 - open/close 590 (time 1785396736.04 total 10.00 last 59.00) total: 900 open/close in 14.83 seconds: 60.69 ops/second llite.lustre-ffff9d1358449800.statahead_max=0 1001 real 0m35.900s user 0m0.038s sys 0m3.337s 'ls -l' 1000 files without statahead: 36 sec llite.lustre-ffff9d1358449800.statahead_max=32 llite.lustre-ffff9d1358449800.statahead_max=32 1001 real 0m23.339s user 0m0.043s sys 0m1.695s 'ls -l' 1000 files with statahead: 23 sec statahead total: 151 statahead wrong: 2 agl total: 143 list_total: 31 fname_total: 104 hit_total: 4750 miss_total: 1696 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 36 seconds llite.lustre-ffff9d1358449800.statahead_stats= statahead total: 151 statahead wrong: 2 agl total: 143 list_total: 31 fname_total: 104 hit_total: 4750 miss_total: 1696 mdc.lustre-MDT0000-mdc-ffff9d1358449800.batch_stats= snapshot_time: 1785396859.411246595 secs.nsecs start_time: 1785396706.532379270 secs.nsecs elapsed_time: 152.878867325 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 5 5 16: 2 5 10 32: 34 89 100 - open/close 454 (time 1785396873.24 total 10.00 last 45.40) - open/close 937 (time 1785396883.25 total 20.01 last 48.27) - open/close 1501 (time 1785396893.28 total 30.03 last 56.24) - open/close 1972 (time 1785396903.28 total 40.04 last 47.08) - open/close 2478 (time 1785396913.28 total 50.04 last 50.60) - open/close 2835 (time 1785396923.36 total 60.11 last 35.43) - open/close 3374 (time 1785396933.37 total 70.13 last 53.82) - open/close 3760 (time 1785396943.38 total 80.13 last 38.59) - open/close 4269 (time 1785396953.45 total 90.20 last 50.54) - open/close 4754 (time 1785396963.45 total 100.21 last 48.48) - open/close 5127 (time 1785396973.48 total 110.23 last 37.21) - open/close 5596 (time 1785396983.48 total 120.23 last 46.90) - open/close 6030 (time 1785396993.51 total 130.26 last 43.26) - open/close 6484 (time 1785397003.52 total 140.27 last 45.36) - open/close 6928 (time 1785397013.53 total 150.29 last 44.34) - open/close 7454 (time 1785397023.54 total 160.30 last 52.52) - open/close 8074 (time 1785397033.54 total 170.30 last 62.00) - open/close 8642 (time 1785397043.58 total 180.33 last 56.61) - open/close 9046 (time 1785397053.59 total 190.34 last 40.36) - open/close 9592 (time 1785397063.59 total 200.35 last 54.59) total: 10000 open/close in 206.55 seconds: 48.41 ops/second llite.lustre-ffff9d1358449800.statahead_batch_max=0 llite.lustre-ffff9d1358449800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d1358449800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m27.244s user 0m0.335s sys 0m16.317s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d1358449800.statahead_batch_max=32 llite.lustre-ffff9d1358449800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d1358449800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d1358449800.stats=clear ls -l files in d123ad.sanity: 10001 real 2m35.677s user 0m0.312s sys 0m15.431s batched statahead RPC= hit count=9986 mdc.lustre-MDT0000-mdc-ffff9d1358449800.batch_stats= snapshot_time: 1785397349.542741772 secs.nsecs start_time: 1785397173.573361251 secs.nsecs elapsed_time: 175.969380521 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9d1358449800.batch_stats=0 llite.lustre-ffff9d1358449800.statahead_max=2048 llite.lustre-ffff9d1358449800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9d1358449800.batch_stats=0 total: 100 open/close in 3.00 seconds: 33.30 ops/second llite.lustre-ffff9d1358449800.statahead_max=0 101 real 0m4.028s user 0m0.008s sys 0m0.390s 'ls -l' 100 files without statahead: 4 sec llite.lustre-ffff9d1358449800.statahead_max=2048 llite.lustre-ffff9d1358449800.statahead_max=2048 101 real 0m1.592s user 0m0.007s sys 0m0.193s 'ls -l' 100 files with statahead: 2 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10085 miss_total: 15 swrong=0 ewrong=0 - open/close 436 (time 1785397803.61 total 10.00 last 43.60) total: 900 open/close in 17.85 seconds: 50.42 ops/second llite.lustre-ffff9d1358449800.statahead_max=0 1001 real 0m34.519s user 0m0.041s sys 0m3.252s 'ls -l' 1000 files without statahead: 35 sec llite.lustre-ffff9d1358449800.statahead_max=2048 llite.lustre-ffff9d1358449800.statahead_max=2048 1001 real 0m11.095s user 0m0.030s sys 0m1.486s 'ls -l' 1000 files with statahead: 11 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 27 seconds llite.lustre-ffff9d1358449800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 mdc.lustre-MDT0000-mdc-ffff9d1358449800.batch_stats= snapshot_time: 1785397900.855238611 secs.nsecs start_time: 1785397353.186777672 secs.nsecs elapsed_time: 547.668460939 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 1 8 66 256: 4 33 100 - open/close 469 (time 1785397914.64 total 10.01 last 46.85) - open/close 918 (time 1785397924.67 total 20.03 last 44.79) - open/close 1534 (time 1785397934.68 total 30.05 last 61.52) - open/close 2122 (time 1785397944.69 total 40.05 last 58.76) - open/close 2673 (time 1785397954.70 total 50.06 last 55.05) - open/close 3163 (time 1785397964.73 total 60.09 last 48.85) - open/close 3636 (time 1785397974.73 total 70.10 last 47.28) - open/close 4013 (time 1785397984.77 total 80.14 last 37.55) - open/close 4473 (time 1785397994.79 total 90.16 last 45.91) - open/close 4897 (time 1785398004.80 total 100.17 last 42.35) - open/close 5325 (time 1785398014.81 total 110.17 last 42.79) - open/close 5755 (time 1785398024.81 total 120.18 last 42.96) - open/close 6330 (time 1785398034.82 total 130.19 last 57.44) - open/close 6915 (time 1785398044.83 total 140.19 last 58.49) - open/close 7617 (time 1785398054.84 total 150.20 last 70.13) - open/close 8048 (time 1785398064.87 total 160.23 last 42.97) - open/close 8544 (time 1785398074.87 total 170.24 last 49.57) - open/close 9188 (time 1785398084.88 total 180.24 last 64.37) - open/close 9708 (time 1785398094.88 total 190.25 last 51.98) total: 10000 open/close in 197.04 seconds: 50.75 ops/second llite.lustre-ffff9d1358449800.statahead_batch_max=0 llite.lustre-ffff9d1358449800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d1358449800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m37.843s user 0m0.379s sys 0m16.518s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d1358449800.statahead_batch_max=256 llite.lustre-ffff9d1358449800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d1358449800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d1358449800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m35.947s user 0m0.339s sys 0m16.115s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9d1358449800.batch_stats= snapshot_time: 1785398337.048057309 secs.nsecs start_time: 1785398216.223899400 secs.nsecs elapsed_time: 120.824157909 secs.nsecs subreqs per batch batches %% cum %% 1: 95 38 38 2: 48 19 58 4: 23 9 68 8: 14 5 73 16: 10 4 77 32: 13 5 83 64: 5 2 85 128: 1 0 85 256: 35 14 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9d1358449800.statahead_batch_max=64 llite.lustre-ffff9d1358449800.statahead_max=128