== sanity test 123ad: Verify batching statahead works correctly ========================================================== 19:44:44 (1788565484) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff95a611238800.batch_stats=0 llite.lustre-ffff95a611238800.statahead_max=32 llite.lustre-ffff95a611238800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff95a611238800.batch_stats=0 total: 100 open/close in 2.40 seconds: 41.61 ops/second llite.lustre-ffff95a611238800.statahead_max=0 101 real 0m2.300s user 0m0.006s sys 0m0.295s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff95a611238800.statahead_max=32 llite.lustre-ffff95a611238800.statahead_max=32 101 real 0m1.194s user 0m0.003s sys 0m0.172s 'ls -l' 100 files with statahead: 1 sec statahead total: 147 statahead wrong: 2 agl total: 141 list_total: 30 fname_total: 106 hit_total: 3978 miss_total: 1391 swrong=2 ewrong=2 - open/close 835 (time 1788565513.45 total 10.01 last 83.39) total: 900 open/close in 10.94 seconds: 82.29 ops/second llite.lustre-ffff95a611238800.statahead_max=0 1001 real 0m18.877s user 0m0.026s sys 0m2.685s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff95a611238800.statahead_max=32 llite.lustre-ffff95a611238800.statahead_max=32 1001 real 0m11.390s user 0m0.029s sys 0m1.328s 'ls -l' 1000 files with statahead: 12 sec statahead total: 148 statahead wrong: 2 agl total: 142 list_total: 31 fname_total: 106 hit_total: 4975 miss_total: 1394 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 29 seconds llite.lustre-ffff95a611238800.statahead_stats= statahead total: 148 statahead wrong: 2 agl total: 142 list_total: 31 fname_total: 106 hit_total: 4975 miss_total: 1394 mdc.lustre-MDT0000-mdc-ffff95a611238800.batch_stats= snapshot_time: 1788565586.438487267 secs.nsecs start_time: 1788565490.337513145 secs.nsecs elapsed_time: 96.100974122 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 579 (time 1788565599.54 total 10.02 last 57.76) - open/close 1215 (time 1788565609.55 total 20.03 last 63.54) - open/close 1925 (time 1788565619.55 total 30.04 last 70.97) - open/close 2581 (time 1788565629.57 total 40.05 last 65.49) - open/close 3074 (time 1788565639.57 total 50.06 last 49.26) - open/close 3566 (time 1788565649.59 total 60.08 last 49.10) - open/close 3910 (time 1788565659.61 total 70.09 last 34.36) - open/close 4321 (time 1788565669.61 total 80.10 last 41.09) - open/close 4842 (time 1788565679.62 total 90.10 last 52.06) - open/close 5502 (time 1788565689.62 total 100.11 last 65.99) - open/close 6076 (time 1788565699.63 total 110.12 last 57.34) - open/close 6649 (time 1788565709.64 total 120.13 last 57.24) - open/close 7091 (time 1788565719.65 total 130.14 last 44.17) - open/close 7533 (time 1788565729.67 total 140.16 last 44.11) - open/close 8275 (time 1788565739.67 total 150.16 last 74.17) - open/close 8966 (time 1788565749.67 total 160.16 last 69.09) - open/close 9378 (time 1788565759.68 total 170.17 last 41.17) - open/close 9899 (time 1788565769.70 total 180.18 last 52.03) total: 10000 open/close in 182.35 seconds: 54.84 ops/second llite.lustre-ffff95a611238800.statahead_batch_max=0 llite.lustre-ffff95a611238800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95a611238800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.063s user 0m0.287s sys 0m12.993s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95a611238800.statahead_batch_max=32 llite.lustre-ffff95a611238800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95a611238800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95a611238800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m54.468s user 0m0.282s sys 0m12.757s batched statahead RPC= hit count=9973 mdc.lustre-MDT0000-mdc-ffff95a611238800.batch_stats= snapshot_time: 1788565974.308569650 secs.nsecs start_time: 1788565848.111277020 secs.nsecs elapsed_time: 126.197292630 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff95a611238800.batch_stats=0 llite.lustre-ffff95a611238800.statahead_max=2048 llite.lustre-ffff95a611238800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff95a611238800.batch_stats=0 total: 100 open/close in 2.91 seconds: 34.33 ops/second llite.lustre-ffff95a611238800.statahead_max=0 101 real 0m1.743s user 0m0.007s sys 0m0.291s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff95a611238800.statahead_max=2048 llite.lustre-ffff95a611238800.statahead_max=2048 101 real 0m1.162s user 0m0.004s sys 0m0.172s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10072 miss_total: 28 swrong=0 ewrong=0 - open/close 768 (time 1788566308.79 total 10.00 last 76.78) total: 900 open/close in 11.66 seconds: 77.16 ops/second llite.lustre-ffff95a611238800.statahead_max=0 1001 real 0m25.287s user 0m0.033s sys 0m2.813s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff95a611238800.statahead_max=2048 llite.lustre-ffff95a611238800.statahead_max=2048 1001 real 0m8.801s user 0m0.031s sys 0m1.498s 'ls -l' 1000 files with statahead: 9 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11071 miss_total: 29 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff95a611238800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11071 miss_total: 29 mdc.lustre-MDT0000-mdc-ffff95a611238800.batch_stats= snapshot_time: 1788566386.038202035 secs.nsecs start_time: 1788565976.845432442 secs.nsecs elapsed_time: 409.192769593 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 669 (time 1788566399.34 total 10.02 last 66.79) - open/close 1437 (time 1788566409.35 total 20.02 last 76.76) - open/close 2029 (time 1788566419.37 total 30.04 last 59.08) - open/close 2771 (time 1788566429.37 total 40.04 last 74.19) - open/close 3629 (time 1788566439.39 total 50.07 last 85.61) - open/close 4280 (time 1788566449.39 total 60.07 last 65.07) - open/close 4949 (time 1788566459.40 total 70.08 last 66.85) - open/close 5660 (time 1788566469.40 total 80.08 last 71.10) - open/close 6360 (time 1788566479.41 total 90.08 last 69.97) - open/close 7076 (time 1788566489.41 total 100.09 last 71.56) - open/close 7807 (time 1788566499.43 total 110.11 last 72.96) - open/close 8699 (time 1788566509.44 total 120.11 last 89.15) - open/close 9440 (time 1788566519.46 total 130.14 last 73.93) total: 10000 open/close in 137.21 seconds: 72.88 ops/second llite.lustre-ffff95a611238800.statahead_batch_max=0 llite.lustre-ffff95a611238800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95a611238800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.225s user 0m0.324s sys 0m13.326s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95a611238800.statahead_batch_max=256 llite.lustre-ffff95a611238800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95a611238800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95a611238800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.607s user 0m0.321s sys 0m13.650s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff95a611238800.batch_stats= snapshot_time: 1788566673.474522836 secs.nsecs start_time: 1788566599.230059154 secs.nsecs elapsed_time: 74.244463682 secs.nsecs subreqs per batch batches %% cum %% 1: 136 43 43 2: 45 14 58 4: 37 11 69 8: 25 8 77 16: 16 5 83 32: 12 3 86 64: 4 1 88 128: 3 0 89 256: 34 10 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff95a611238800.statahead_batch_max=64 llite.lustre-ffff95a611238800.statahead_max=128