== sanity test 123ad: Verify batching statahead works correctly ========================================================== 15:55:50 (1783972550) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8d21853bf800.batch_stats=0 llite.lustre-ffff8d21853bf800.statahead_max=32 llite.lustre-ffff8d21853bf800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8d21853bf800.batch_stats=0 total: 100 open/close in 3.00 seconds: 33.28 ops/second llite.lustre-ffff8d21853bf800.statahead_max=0 101 real 0m2.849s user 0m0.003s sys 0m0.342s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8d21853bf800.statahead_max=32 llite.lustre-ffff8d21853bf800.statahead_max=32 101 real 0m0.947s user 0m0.007s sys 0m0.208s 'ls -l' 100 files with statahead: 1 sec statahead total: 119 statahead wrong: 2 agl total: 113 list_total: 30 fname_total: 69 hit_total: 3699 miss_total: 1436 swrong=2 ewrong=2 - open/close 649 (time 1783972579.47 total 10.01 last 64.81) total: 900 open/close in 13.78 seconds: 65.33 ops/second llite.lustre-ffff8d21853bf800.statahead_max=0 1001 real 0m22.772s user 0m0.031s sys 0m3.378s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff8d21853bf800.statahead_max=32 llite.lustre-ffff8d21853bf800.statahead_max=32 1001 real 0m11.141s user 0m0.031s sys 0m1.606s 'ls -l' 1000 files with statahead: 11 sec statahead total: 120 statahead wrong: 2 agl total: 114 list_total: 31 fname_total: 69 hit_total: 4698 miss_total: 1437 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff8d21853bf800.statahead_stats= statahead total: 120 statahead wrong: 2 agl total: 114 list_total: 31 fname_total: 69 hit_total: 4698 miss_total: 1437 mdc.lustre-MDT0000-mdc-ffff8d21853bf800.batch_stats= snapshot_time: 1783972651.739048061 secs.nsecs start_time: 1783972555.528815086 secs.nsecs elapsed_time: 96.210232975 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 2 2 8: 2 5 7 16: 2 5 12 32: 34 87 100 - open/close 661 (time 1783972664.47 total 10.01 last 66.06) - open/close 1318 (time 1783972674.48 total 20.01 last 65.66) - open/close 1991 (time 1783972684.48 total 30.01 last 67.29) - open/close 2640 (time 1783972694.49 total 40.03 last 64.82) - open/close 3263 (time 1783972704.50 total 50.04 last 62.25) - open/close 3945 (time 1783972714.52 total 60.05 last 68.10) - open/close 4563 (time 1783972724.52 total 70.06 last 61.77) - open/close 5202 (time 1783972734.52 total 80.06 last 63.88) - open/close 5836 (time 1783972744.53 total 90.06 last 63.37) - open/close 6492 (time 1783972754.54 total 100.08 last 65.52) - open/close 7111 (time 1783972764.55 total 110.08 last 61.86) - open/close 7811 (time 1783972774.56 total 120.09 last 69.94) - open/close 8448 (time 1783972784.56 total 130.09 last 63.70) - open/close 9108 (time 1783972794.58 total 140.11 last 65.86) - open/close 9752 (time 1783972804.60 total 150.13 last 64.28) total: 10000 open/close in 154.67 seconds: 64.65 ops/second llite.lustre-ffff8d21853bf800.statahead_batch_max=0 llite.lustre-ffff8d21853bf800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d21853bf800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.724s user 0m0.342s sys 0m16.895s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8d21853bf800.statahead_batch_max=32 llite.lustre-ffff8d21853bf800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d21853bf800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8d21853bf800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m57.477s user 0m0.333s sys 0m16.229s batched statahead RPC= hit count=9980 mdc.lustre-MDT0000-mdc-ffff8d21853bf800.batch_stats= snapshot_time: 1783973027.755376226 secs.nsecs start_time: 1783972892.161033904 secs.nsecs elapsed_time: 135.594342322 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8d21853bf800.batch_stats=0 llite.lustre-ffff8d21853bf800.statahead_max=2048 llite.lustre-ffff8d21853bf800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8d21853bf800.batch_stats=0 total: 100 open/close in 2.92 seconds: 34.26 ops/second llite.lustre-ffff8d21853bf800.statahead_max=0 101 real 0m2.699s user 0m0.010s sys 0m0.329s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8d21853bf800.statahead_max=2048 llite.lustre-ffff8d21853bf800.statahead_max=2048 101 real 0m1.067s user 0m0.007s sys 0m0.214s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10079 miss_total: 21 swrong=0 ewrong=0 - open/close 579 (time 1783973385.83 total 10.01 last 57.86) total: 900 open/close in 16.72 seconds: 53.81 ops/second llite.lustre-ffff8d21853bf800.statahead_max=0 1001 real 0m30.314s user 0m0.043s sys 0m3.649s 'ls -l' 1000 files without statahead: 30 sec llite.lustre-ffff8d21853bf800.statahead_max=2048 llite.lustre-ffff8d21853bf800.statahead_max=2048 1001 real 0m7.672s user 0m0.028s sys 0m1.685s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff8d21853bf800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 mdc.lustre-MDT0000-mdc-ffff8d21853bf800.batch_stats= snapshot_time: 1783973466.580923058 secs.nsecs start_time: 1783973029.904829361 secs.nsecs elapsed_time: 436.676093697 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 1 8 66 256: 4 33 100 - open/close 602 (time 1783973479.04 total 10.01 last 60.17) - open/close 1244 (time 1783973489.06 total 20.02 last 64.07) - open/close 1920 (time 1783973499.07 total 30.03 last 67.57) - open/close 2566 (time 1783973509.08 total 40.05 last 64.49) - open/close 3244 (time 1783973519.09 total 50.05 last 67.75) - open/close 3913 (time 1783973529.09 total 60.06 last 66.88) - open/close 4566 (time 1783973539.10 total 70.06 last 65.28) - open/close 5241 (time 1783973549.10 total 80.06 last 67.49) - open/close 5895 (time 1783973559.11 total 90.08 last 65.31) - open/close 6624 (time 1783973569.12 total 100.08 last 72.83) - open/close 7321 (time 1783973579.13 total 110.09 last 69.66) - open/close 7923 (time 1783973589.13 total 120.10 last 60.17) - open/close 8598 (time 1783973599.14 total 130.11 last 67.43) - open/close 9283 (time 1783973609.15 total 140.11 last 68.46) total: 10000 open/close in 149.45 seconds: 66.91 ops/second llite.lustre-ffff8d21853bf800.statahead_batch_max=0 llite.lustre-ffff8d21853bf800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d21853bf800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.406s user 0m0.373s sys 0m18.755s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8d21853bf800.statahead_batch_max=256 llite.lustre-ffff8d21853bf800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8d21853bf800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8d21853bf800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.111s user 0m0.319s sys 0m16.999s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8d21853bf800.batch_stats= snapshot_time: 1783973762.692148267 secs.nsecs start_time: 1783973690.760495269 secs.nsecs elapsed_time: 71.931652998 secs.nsecs subreqs per batch batches %% cum %% 1: 223 53 53 2: 55 13 67 4: 46 11 78 8: 22 5 83 16: 14 3 87 32: 15 3 90 64: 4 0 91 128: 0 0 91 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8d21853bf800.statahead_batch_max=64 llite.lustre-ffff8d21853bf800.statahead_max=128