== sanity test 123ad: Verify batching statahead works correctly ========================================================== 23:32:51 (1784086371) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9d170621c800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9d170621c800.batch_stats=0 llite.lustre-ffff9d170621c800.statahead_max=32 llite.lustre-ffff9d170621c800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9d170621c800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9d170621c800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 0.60 seconds: 165.85 ops/second llite.lustre-ffff9d170621c800.statahead_max=0 101 real 0m0.581s user 0m0.003s sys 0m0.224s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff9d170621c800.statahead_max=32 llite.lustre-ffff9d170621c800.statahead_max=32 101 real 0m0.529s user 0m0.005s sys 0m0.346s 'ls -l' 100 files with statahead: 0 sec statahead total: 59 statahead wrong: 1 agl total: 55 list_total: 25 fname_total: 34 hit_total: 13588 miss_total: 194 swrong=1 ewrong=1 total: 900 open/close in 1.89 seconds: 475.26 ops/second llite.lustre-ffff9d170621c800.statahead_max=0 1001 real 0m5.899s user 0m0.013s sys 0m2.286s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff9d170621c800.statahead_max=32 llite.lustre-ffff9d170621c800.statahead_max=32 1001 real 0m3.899s user 0m0.022s sys 0m1.909s 'ls -l' 1000 files with statahead: 4 sec statahead total: 60 statahead wrong: 1 agl total: 56 list_total: 26 fname_total: 34 hit_total: 14587 miss_total: 195 swrong=1 ewrong=1 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 6 seconds llite.lustre-ffff9d170621c800.statahead_stats= statahead total: 60 statahead wrong: 1 agl total: 56 list_total: 26 fname_total: 34 hit_total: 14587 miss_total: 195 mdc.lustre-MDT0000-mdc-ffff9d170621c800.batch_stats= snapshot_time: 1784086398.260697866 secs.nsecs start_time: 1784086373.158381581 secs.nsecs elapsed_time: 25.102316285 secs.nsecs subreqs per batch batches %% cum %% 1: 2 5 5 2: 3 7 13 4: 1 2 15 8: 3 7 23 16: 23 60 84 32: 6 15 100 mdc.lustre-MDT0001-mdc-ffff9d170621c800.batch_stats= snapshot_time: 1784086398.260771454 secs.nsecs start_time: 1784086373.158438996 secs.nsecs elapsed_time: 25.102332458 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 1 2 5 8: 4 10 15 16: 9 23 38 32: 24 61 100 - open/close 4850 (time 1784086409.03 total 10.00 last 484.96) - open/close 9567 (time 1784086419.03 total 20.00 last 471.68) total: 10000 open/close in 20.92 seconds: 478.00 ops/second llite.lustre-ffff9d170621c800.statahead_batch_max=0 llite.lustre-ffff9d170621c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d170621c800.stats=clear mdc.lustre-MDT0001-mdc-ffff9d170621c800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m34.956s user 0m0.139s sys 0m33.301s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d170621c800.statahead_batch_max=32 llite.lustre-ffff9d170621c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d170621c800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9d170621c800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d170621c800.stats=clear mdc.lustre-MDT0001-mdc-ffff9d170621c800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m39.255s user 0m0.159s sys 0m22.794s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9d170621c800.batch_stats= snapshot_time: 1784086503.317260135 secs.nsecs start_time: 1784086457.965283084 secs.nsecs elapsed_time: 45.351977051 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff9d170621c800.batch_stats= snapshot_time: 1784086503.317328661 secs.nsecs start_time: 1784086457.965365802 secs.nsecs elapsed_time: 45.351962859 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9d170621c800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9d170621c800.batch_stats=0 llite.lustre-ffff9d170621c800.statahead_max=2048 llite.lustre-ffff9d170621c800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9d170621c800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9d170621c800.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.59 seconds: 168.27 ops/second llite.lustre-ffff9d170621c800.statahead_max=0 101 real 0m0.581s user 0m0.001s sys 0m0.220s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff9d170621c800.statahead_max=2048 llite.lustre-ffff9d170621c800.statahead_max=2048 101 real 0m0.388s user 0m0.002s sys 0m0.245s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 swrong=0 ewrong=0 total: 900 open/close in 2.22 seconds: 405.06 ops/second llite.lustre-ffff9d170621c800.statahead_max=0 1001 real 0m5.657s user 0m0.014s sys 0m2.088s 'ls -l' 1000 files without statahead: 5 sec llite.lustre-ffff9d170621c800.statahead_max=2048 llite.lustre-ffff9d170621c800.statahead_max=2048 1001 real 0m3.290s user 0m0.012s sys 0m1.252s 'ls -l' 1000 files with statahead: 3 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 swrong=0 ewrong=0 - open/close 4330 (time 1784086625.64 total 10.00 last 432.98) total: 9000 open/close in 19.64 seconds: 458.33 ops/second llite.lustre-ffff9d170621c800.statahead_max=0 10001 real 0m54.947s user 0m0.142s sys 0m19.764s 'ls -l' 10000 files without statahead: 55 sec llite.lustre-ffff9d170621c800.statahead_max=2048 llite.lustre-ffff9d170621c800.statahead_max=2048 10001 real 0m28.386s user 0m0.133s sys 0m19.454s 'ls -l' 10000 files with statahead: 28 sec statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 21096 miss_total: 4 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 85 seconds llite.lustre-ffff9d170621c800.statahead_stats= statahead total: 4 statahead wrong: 0 agl total: 4 list_total: 4 fname_total: 0 hit_total: 21096 miss_total: 4 mdc.lustre-MDT0000-mdc-ffff9d170621c800.batch_stats= snapshot_time: 1784086816.039970204 secs.nsecs start_time: 1784086504.146837223 secs.nsecs elapsed_time: 311.893132981 secs.nsecs subreqs per batch batches %% cum %% 1: 101 59 59 2: 16 9 69 4: 15 8 78 8: 4 2 80 16: 0 0 80 32: 0 0 80 64: 0 0 80 128: 9 5 85 256: 24 14 100 mdc.lustre-MDT0001-mdc-ffff9d170621c800.batch_stats= snapshot_time: 1784086816.040011836 secs.nsecs start_time: 1784086504.146988676 secs.nsecs elapsed_time: 311.893023160 secs.nsecs subreqs per batch batches %% cum %% 1: 103 59 59 2: 20 11 71 4: 8 4 76 8: 5 2 79 16: 0 0 79 32: 0 0 79 64: 1 0 79 128: 6 3 83 256: 29 16 100 - open/close 4779 (time 1784086826.94 total 10.00 last 477.83) - open/close 9695 (time 1784086836.94 total 20.00 last 491.50) total: 10000 open/close in 20.62 seconds: 485.08 ops/second llite.lustre-ffff9d170621c800.statahead_batch_max=0 llite.lustre-ffff9d170621c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d170621c800.stats=clear mdc.lustre-MDT0001-mdc-ffff9d170621c800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m33.137s user 0m0.139s sys 0m28.754s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d170621c800.statahead_batch_max=256 llite.lustre-ffff9d170621c800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d170621c800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9d170621c800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d170621c800.stats=clear mdc.lustre-MDT0001-mdc-ffff9d170621c800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m28.374s user 0m0.133s sys 0m20.352s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9d170621c800.batch_stats= snapshot_time: 1784086907.839010300 secs.nsecs start_time: 1784086873.383642397 secs.nsecs elapsed_time: 34.455367903 secs.nsecs subreqs per batch batches %% cum %% 1: 403 66 66 2: 88 14 81 4: 44 7 88 8: 9 1 90 16: 3 0 90 32: 21 3 94 64: 2 0 94 128: 0 0 94 256: 34 5 100 mdc.lustre-MDT0001-mdc-ffff9d170621c800.batch_stats= snapshot_time: 1784086907.839052902 secs.nsecs start_time: 1784086873.383693135 secs.nsecs elapsed_time: 34.455359767 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9d170621c800.statahead_batch_max=64 llite.lustre-ffff9d170621c800.statahead_max=128