== sanity test 123ad: Verify batching statahead works correctly ========================================================== 12:22:38 (1779207758) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.batch_stats=0 llite.lustre-ffff9951c4cf5800.statahead_max=32 llite.lustre-ffff9951c4cf5800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.batch_stats=0 total: 100 open/close in 3.29 seconds: 30.40 ops/second llite.lustre-ffff9951c4cf5800.statahead_max=0 101 real 0m2.541s user 0m0.009s sys 0m0.320s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9951c4cf5800.statahead_max=32 llite.lustre-ffff9951c4cf5800.statahead_max=32 101 real 0m0.899s user 0m0.001s sys 0m0.177s 'ls -l' 100 files with statahead: 1 sec statahead total: 114 statahead wrong: 2 agl total: 108 list_total: 29 fname_total: 74 hit_total: 3883 miss_total: 1479 swrong=2 ewrong=2 - open/close 639 (time 1779207787.52 total 10.01 last 63.86) total: 900 open/close in 13.40 seconds: 67.17 ops/second llite.lustre-ffff9951c4cf5800.statahead_max=0 1001 real 0m20.474s user 0m0.037s sys 0m2.942s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff9951c4cf5800.statahead_max=32 llite.lustre-ffff9951c4cf5800.statahead_max=32 1001 real 0m9.681s user 0m0.029s sys 0m1.437s 'ls -l' 1000 files with statahead: 9 sec statahead total: 115 statahead wrong: 2 agl total: 109 list_total: 30 fname_total: 74 hit_total: 4880 miss_total: 1482 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff9951c4cf5800.statahead_stats= statahead total: 115 statahead wrong: 2 agl total: 109 list_total: 30 fname_total: 74 hit_total: 4880 miss_total: 1482 mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.batch_stats= snapshot_time: 1779207852.215652267 secs.nsecs start_time: 1779207763.989211660 secs.nsecs elapsed_time: 88.226440607 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 760 (time 1779207864.39 total 10.00 last 75.99) - open/close 1532 (time 1779207874.40 total 20.00 last 77.18) - open/close 2250 (time 1779207884.40 total 30.00 last 71.80) - open/close 2934 (time 1779207894.40 total 40.01 last 68.39) - open/close 3621 (time 1779207904.41 total 50.02 last 68.62) - open/close 4270 (time 1779207914.42 total 60.02 last 64.87) - open/close 4917 (time 1779207924.42 total 70.03 last 64.68) - open/close 5489 (time 1779207934.42 total 80.03 last 57.18) - open/close 6070 (time 1779207944.43 total 90.04 last 58.07) - open/close 6812 (time 1779207954.43 total 100.04 last 74.19) - open/close 7469 (time 1779207964.44 total 110.04 last 65.66) - open/close 8119 (time 1779207974.44 total 120.05 last 64.96) - open/close 8726 (time 1779207984.44 total 130.05 last 60.70) - open/close 9391 (time 1779207994.45 total 140.05 last 66.47) total: 10000 open/close in 148.41 seconds: 67.38 ops/second llite.lustre-ffff9951c4cf5800.statahead_batch_max=0 llite.lustre-ffff9951c4cf5800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.141s user 0m0.329s sys 0m16.829s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9951c4cf5800.statahead_batch_max=32 llite.lustre-ffff9951c4cf5800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m47.540s user 0m0.309s sys 0m14.713s batched statahead RPC= hit count=9979 mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.batch_stats= snapshot_time: 1779208190.012298263 secs.nsecs start_time: 1779208069.585025039 secs.nsecs elapsed_time: 120.427273224 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.batch_stats=0 llite.lustre-ffff9951c4cf5800.statahead_max=2048 llite.lustre-ffff9951c4cf5800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.batch_stats=0 total: 100 open/close in 2.57 seconds: 38.90 ops/second llite.lustre-ffff9951c4cf5800.statahead_max=0 101 real 0m2.879s user 0m0.004s sys 0m0.324s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9951c4cf5800.statahead_max=2048 llite.lustre-ffff9951c4cf5800.statahead_max=2048 101 real 0m0.998s user 0m0.007s sys 0m0.164s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10078 miss_total: 22 swrong=0 ewrong=0 - open/close 670 (time 1779208526.77 total 10.01 last 66.94) total: 900 open/close in 13.66 seconds: 65.88 ops/second llite.lustre-ffff9951c4cf5800.statahead_max=0 1001 real 0m26.681s user 0m0.038s sys 0m3.233s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff9951c4cf5800.statahead_max=2048 llite.lustre-ffff9951c4cf5800.statahead_max=2048 1001 real 0m10.068s user 0m0.033s sys 0m1.376s 'ls -l' 1000 files with statahead: 10 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9951c4cf5800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.batch_stats= snapshot_time: 1779208605.121111120 secs.nsecs start_time: 1779208192.817652214 secs.nsecs elapsed_time: 412.303458906 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 645 (time 1779208617.75 total 10.01 last 64.42) - open/close 1346 (time 1779208627.75 total 20.02 last 70.08) - open/close 2075 (time 1779208637.76 total 30.02 last 72.85) - open/close 2768 (time 1779208647.77 total 40.03 last 69.25) - open/close 3554 (time 1779208657.78 total 50.04 last 78.51) - open/close 4224 (time 1779208667.78 total 60.05 last 66.96) - open/close 4987 (time 1779208677.79 total 70.05 last 76.29) - open/close 5709 (time 1779208687.79 total 80.05 last 72.16) - open/close 6473 (time 1779208697.80 total 90.06 last 76.35) - open/close 7208 (time 1779208707.81 total 100.07 last 73.43) - open/close 7972 (time 1779208717.82 total 110.08 last 76.29) - open/close 8591 (time 1779208727.83 total 120.09 last 61.85) - open/close 9312 (time 1779208737.83 total 130.10 last 72.06) - open/close 9927 (time 1779208747.83 total 140.10 last 61.50) total: 10000 open/close in 141.34 seconds: 70.75 ops/second llite.lustre-ffff9951c4cf5800.statahead_batch_max=0 llite.lustre-ffff9951c4cf5800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.184s user 0m0.322s sys 0m15.089s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9951c4cf5800.statahead_batch_max=256 llite.lustre-ffff9951c4cf5800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.807s user 0m0.298s sys 0m15.464s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9951c4cf5800.batch_stats= snapshot_time: 1779208907.488597614 secs.nsecs start_time: 1779208830.188195018 secs.nsecs elapsed_time: 77.300402596 secs.nsecs subreqs per batch batches % cum % 1: 178 46 46 2: 60 15 62 4: 48 12 74 8: 26 6 81 16: 14 3 85 32: 21 5 90 64: 1 0 91 128: 0 0 91 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9951c4cf5800.statahead_batch_max=64 llite.lustre-ffff9951c4cf5800.statahead_max=128