== sanity test 123ad: Verify batching statahead works correctly ========================================================== 14:28:52 (1780856932) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8b070860a800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8b070860a800.batch_stats=0 llite.lustre-ffff8b070860a800.statahead_max=32 llite.lustre-ffff8b070860a800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8b070860a800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8b070860a800.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.92 seconds: 52.05 ops/second llite.lustre-ffff8b070860a800.statahead_max=0 101 real 0m1.851s user 0m0.009s sys 0m0.340s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8b070860a800.statahead_max=32 llite.lustre-ffff8b070860a800.statahead_max=32 101 real 0m0.670s user 0m0.004s sys 0m0.177s 'ls -l' 100 files with statahead: 1 sec statahead total: 156 statahead wrong: 2 agl total: 149 list_total: 31 fname_total: 119 hit_total: 3955 miss_total: 1215 swrong=2 ewrong=2 total: 900 open/close in 8.59 seconds: 104.79 ops/second llite.lustre-ffff8b070860a800.statahead_max=0 1001 real 0m15.928s user 0m0.039s sys 0m3.118s 'ls -l' 1000 files without statahead: 16 sec llite.lustre-ffff8b070860a800.statahead_max=32 llite.lustre-ffff8b070860a800.statahead_max=32 1001 real 0m5.516s user 0m0.022s sys 0m1.727s 'ls -l' 1000 files with statahead: 6 sec statahead total: 157 statahead wrong: 2 agl total: 150 list_total: 32 fname_total: 119 hit_total: 4954 miss_total: 1216 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 13 seconds llite.lustre-ffff8b070860a800.statahead_stats= statahead total: 157 statahead wrong: 2 agl total: 150 list_total: 32 fname_total: 119 hit_total: 4954 miss_total: 1216 mdc.lustre-MDT0000-mdc-ffff8b070860a800.batch_stats= snapshot_time: 1780856993.628804987 secs.nsecs start_time: 1780856935.436070116 secs.nsecs elapsed_time: 58.192734871 secs.nsecs subreqs per batch batches % cum % 1: 3 8 8 2: 2 5 13 4: 1 2 16 8: 1 2 18 16: 24 64 83 32: 6 16 100 mdc.lustre-MDT0001-mdc-ffff8b070860a800.batch_stats= snapshot_time: 1780856993.628868393 secs.nsecs start_time: 1780856935.436146973 secs.nsecs elapsed_time: 58.192721420 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 1 2 2 8: 3 7 10 16: 12 31 42 32: 22 57 100 - open/close 1119 (time 1780857004.98 total 10.01 last 111.83) - open/close 2342 (time 1780857014.98 total 20.01 last 122.27) - open/close 3570 (time 1780857024.99 total 30.01 last 122.73) - open/close 4700 (time 1780857034.99 total 40.02 last 112.94) - open/close 5849 (time 1780857045.00 total 50.03 last 114.82) - open/close 6955 (time 1780857055.01 total 60.04 last 110.47) - open/close 8177 (time 1780857065.02 total 70.04 last 122.17) - open/close 9369 (time 1780857075.02 total 80.04 last 119.17) total: 10000 open/close in 85.53 seconds: 116.92 ops/second llite.lustre-ffff8b070860a800.statahead_batch_max=0 llite.lustre-ffff8b070860a800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b070860a800.stats=clear mdc.lustre-MDT0001-mdc-ffff8b070860a800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m42.517s user 0m0.259s sys 0m23.453s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b070860a800.statahead_batch_max=32 llite.lustre-ffff8b070860a800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b070860a800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8b070860a800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b070860a800.stats=clear mdc.lustre-MDT0001-mdc-ffff8b070860a800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.505s user 0m0.226s sys 0m17.719s batched statahead RPC= hit count=9988 mdc.lustre-MDT0000-mdc-ffff8b070860a800.batch_stats= snapshot_time: 1780857194.187137337 secs.nsecs start_time: 1780857128.905152405 secs.nsecs elapsed_time: 65.281984932 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff8b070860a800.batch_stats= snapshot_time: 1780857194.187199342 secs.nsecs start_time: 1780857128.905282640 secs.nsecs elapsed_time: 65.281916702 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8b070860a800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8b070860a800.batch_stats=0 llite.lustre-ffff8b070860a800.statahead_max=2048 llite.lustre-ffff8b070860a800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8b070860a800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8b070860a800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.54 seconds: 64.85 ops/second llite.lustre-ffff8b070860a800.statahead_max=0 101 real 0m1.285s user 0m0.005s sys 0m0.315s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff8b070860a800.statahead_max=2048 llite.lustre-ffff8b070860a800.statahead_max=2048 101 real 0m0.444s user 0m0.002s sys 0m0.210s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10087 miss_total: 13 swrong=0 ewrong=0 total: 900 open/close in 7.01 seconds: 128.35 ops/second llite.lustre-ffff8b070860a800.statahead_max=0 1001 real 0m13.488s user 0m0.030s sys 0m2.974s 'ls -l' 1000 files without statahead: 13 sec llite.lustre-ffff8b070860a800.statahead_max=2048 llite.lustre-ffff8b070860a800.statahead_max=2048 1001 real 0m3.552s user 0m0.018s sys 0m1.656s 'ls -l' 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 10 seconds llite.lustre-ffff8b070860a800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11086 miss_total: 14 mdc.lustre-MDT0000-mdc-ffff8b070860a800.batch_stats= snapshot_time: 1780857396.044712628 secs.nsecs start_time: 1780857195.276455363 secs.nsecs elapsed_time: 200.768257265 secs.nsecs subreqs per batch batches % cum % 1: 3 37 37 2: 1 12 50 4: 1 12 62 8: 1 12 75 16: 0 0 75 32: 0 0 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff8b070860a800.batch_stats= snapshot_time: 1780857396.044779653 secs.nsecs start_time: 1780857195.276583776 secs.nsecs elapsed_time: 200.768195877 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 1 10 10 8: 3 30 40 16: 2 20 60 32: 1 10 70 64: 1 10 80 128: 0 0 80 256: 2 20 100 - open/close 1334 (time 1780857407.40 total 10.00 last 133.40) - open/close 2941 (time 1780857417.41 total 20.00 last 160.65) - open/close 4485 (time 1780857427.41 total 30.00 last 154.39) - open/close 5746 (time 1780857437.41 total 40.01 last 126.06) - open/close 7249 (time 1780857447.42 total 50.02 last 150.17) - open/close 8727 (time 1780857457.42 total 60.02 last 147.75) total: 10000 open/close in 67.63 seconds: 147.86 ops/second llite.lustre-ffff8b070860a800.statahead_batch_max=0 llite.lustre-ffff8b070860a800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b070860a800.stats=clear mdc.lustre-MDT0001-mdc-ffff8b070860a800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m35.567s user 0m0.267s sys 0m27.584s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8b070860a800.statahead_batch_max=256 llite.lustre-ffff8b070860a800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8b070860a800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8b070860a800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8b070860a800.stats=clear mdc.lustre-MDT0001-mdc-ffff8b070860a800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m32.643s user 0m0.265s sys 0m21.741s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8b070860a800.batch_stats= snapshot_time: 1780857544.573240606 secs.nsecs start_time: 1780857504.672539910 secs.nsecs elapsed_time: 39.900700696 secs.nsecs subreqs per batch batches % cum % 1: 417 66 66 2: 86 13 80 4: 49 7 87 8: 13 2 89 16: 8 1 91 32: 20 3 94 64: 1 0 94 128: 1 0 94 256: 33 5 100 mdc.lustre-MDT0001-mdc-ffff8b070860a800.batch_stats= snapshot_time: 1780857544.573310431 secs.nsecs start_time: 1780857504.672625442 secs.nsecs elapsed_time: 39.900684989 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8b070860a800.statahead_batch_max=64 llite.lustre-ffff8b070860a800.statahead_max=128