== sanity test 123ad: Verify batching statahead works correctly ========================================================== 06:47:42 (1789123662) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.batch_stats=0 llite.lustre-ffff9ba52edc5000.statahead_max=32 llite.lustre-ffff9ba52edc5000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.batch_stats=0 total: 100 open/close in 3.28 seconds: 30.51 ops/second llite.lustre-ffff9ba52edc5000.statahead_max=0 101 real 0m2.732s user 0m0.008s sys 0m0.322s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9ba52edc5000.statahead_max=32 llite.lustre-ffff9ba52edc5000.statahead_max=32 101 real 0m1.299s user 0m0.005s sys 0m0.180s 'ls -l' 100 files with statahead: 1 sec statahead total: 123 statahead wrong: 2 agl total: 117 list_total: 31 fname_total: 79 hit_total: 3567 miss_total: 1517 swrong=2 ewrong=2 - open/close 579 (time 1789123692.99 total 10.00 last 57.88) total: 900 open/close in 16.22 seconds: 55.49 ops/second llite.lustre-ffff9ba52edc5000.statahead_max=0 1001 real 0m27.446s user 0m0.039s sys 0m3.087s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff9ba52edc5000.statahead_max=32 llite.lustre-ffff9ba52edc5000.statahead_max=32 1001 real 0m12.394s user 0m0.032s sys 0m1.427s 'ls -l' 1000 files with statahead: 12 sec statahead total: 124 statahead wrong: 2 agl total: 118 list_total: 32 fname_total: 79 hit_total: 4561 miss_total: 1523 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff9ba52edc5000.statahead_stats= statahead total: 124 statahead wrong: 2 agl total: 118 list_total: 32 fname_total: 79 hit_total: 4561 miss_total: 1523 mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.batch_stats= snapshot_time: 1789123782.730780656 secs.nsecs start_time: 1789123667.612125957 secs.nsecs elapsed_time: 115.118654699 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 2 5 7 16: 2 5 12 32: 34 87 100 - open/close 577 (time 1789123796.74 total 10.00 last 57.69) - open/close 1132 (time 1789123806.75 total 20.01 last 55.47) - open/close 1728 (time 1789123816.76 total 30.02 last 59.54) - open/close 2206 (time 1789123826.76 total 40.02 last 47.80) - open/close 2790 (time 1789123836.77 total 50.03 last 58.34) - open/close 3332 (time 1789123846.77 total 60.03 last 54.19) - open/close 3910 (time 1789123856.79 total 70.05 last 57.69) - open/close 4486 (time 1789123866.80 total 80.06 last 57.54) - open/close 5114 (time 1789123876.80 total 90.06 last 62.78) - open/close 5762 (time 1789123886.80 total 100.06 last 64.80) - open/close 6389 (time 1789123896.81 total 110.07 last 62.66) - open/close 7008 (time 1789123906.82 total 120.08 last 61.83) - open/close 7577 (time 1789123916.83 total 130.09 last 56.82) - open/close 8162 (time 1789123926.84 total 140.10 last 58.45) - open/close 8760 (time 1789123936.85 total 150.11 last 59.74) - open/close 9171 (time 1789123946.87 total 160.13 last 41.01) - open/close 9666 (time 1789123956.90 total 170.16 last 49.36) total: 10000 open/close in 175.33 seconds: 57.03 ops/second llite.lustre-ffff9ba52edc5000.statahead_batch_max=0 llite.lustre-ffff9ba52edc5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m19.719s user 0m0.316s sys 0m13.925s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ba52edc5000.statahead_batch_max=32 llite.lustre-ffff9ba52edc5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m51.844s user 0m0.314s sys 0m13.530s batched statahead RPC= hit count=9976 mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.batch_stats= snapshot_time: 1789124177.969299118 secs.nsecs start_time: 1789124052.407719137 secs.nsecs elapsed_time: 125.561579981 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.batch_stats=0 llite.lustre-ffff9ba52edc5000.statahead_max=2048 llite.lustre-ffff9ba52edc5000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.batch_stats=0 total: 100 open/close in 2.78 seconds: 35.93 ops/second llite.lustre-ffff9ba52edc5000.statahead_max=0 101 real 0m2.806s user 0m0.005s sys 0m0.302s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9ba52edc5000.statahead_max=2048 llite.lustre-ffff9ba52edc5000.statahead_max=2048 101 real 0m1.001s user 0m0.006s sys 0m0.187s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10075 miss_total: 25 swrong=0 ewrong=0 - open/close 642 (time 1789124539.39 total 10.00 last 64.19) total: 900 open/close in 15.34 seconds: 58.66 ops/second llite.lustre-ffff9ba52edc5000.statahead_max=0 1001 real 0m24.504s user 0m0.030s sys 0m2.981s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff9ba52edc5000.statahead_max=2048 llite.lustre-ffff9ba52edc5000.statahead_max=2048 1001 real 0m8.367s user 0m0.022s sys 0m1.309s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff9ba52edc5000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.batch_stats= snapshot_time: 1789124617.998702755 secs.nsecs start_time: 1789124180.814546504 secs.nsecs elapsed_time: 437.184156251 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 0 0 40 64: 1 10 50 128: 1 10 60 256: 4 40 100 - open/close 666 (time 1789124630.62 total 10.01 last 66.55) - open/close 1332 (time 1789124640.62 total 20.01 last 66.55) - open/close 1993 (time 1789124650.63 total 30.02 last 66.04) - open/close 2655 (time 1789124660.67 total 40.06 last 65.97) - open/close 3209 (time 1789124670.68 total 50.07 last 55.32) - open/close 3697 (time 1789124680.68 total 60.08 last 48.79) - open/close 4282 (time 1789124690.70 total 70.09 last 58.39) - open/close 4933 (time 1789124700.70 total 80.09 last 65.10) - open/close 5583 (time 1789124710.70 total 90.09 last 65.00) - open/close 6075 (time 1789124720.70 total 100.10 last 49.19) - open/close 6615 (time 1789124730.72 total 110.12 last 53.89) - open/close 7297 (time 1789124740.73 total 120.12 last 68.18) - open/close 7952 (time 1789124750.73 total 130.13 last 65.45) - open/close 8565 (time 1789124760.74 total 140.13 last 61.30) - open/close 9258 (time 1789124770.74 total 150.14 last 69.24) - open/close 9884 (time 1789124780.75 total 160.14 last 62.58) total: 10000 open/close in 161.56 seconds: 61.90 ops/second llite.lustre-ffff9ba52edc5000.statahead_batch_max=0 llite.lustre-ffff9ba52edc5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.533s user 0m0.297s sys 0m13.427s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ba52edc5000.statahead_batch_max=256 llite.lustre-ffff9ba52edc5000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.310s user 0m0.326s sys 0m13.548s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9ba52edc5000.batch_stats= snapshot_time: 1789124921.197068568 secs.nsecs start_time: 1789124858.498364880 secs.nsecs elapsed_time: 62.698703688 secs.nsecs subreqs per batch batches %% cum %% 1: 106 42 42 2: 35 14 57 4: 22 8 65 8: 14 5 71 16: 15 6 77 32: 15 6 83 64: 5 2 85 128: 0 0 85 256: 35 14 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9ba52edc5000.statahead_batch_max=64 llite.lustre-ffff9ba52edc5000.statahead_max=128