== sanity test 123ad: Verify batching statahead works correctly ========================================================== 03:31:17 (1789371077) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff958a09559000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff958a09559000.batch_stats=0 llite.lustre-ffff958a09559000.statahead_max=32 llite.lustre-ffff958a09559000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff958a09559000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff958a09559000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.31 seconds: 76.62 ops/second llite.lustre-ffff958a09559000.statahead_max=0 101 real 0m1.252s user 0m0.003s sys 0m0.293s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff958a09559000.statahead_max=32 llite.lustre-ffff958a09559000.statahead_max=32 101 real 0m0.512s user 0m0.004s sys 0m0.174s 'ls -l' 100 files with statahead: 0 sec statahead total: 81 statahead wrong: 1 agl total: 76 list_total: 30 fname_total: 50 hit_total: 13905 miss_total: 244 swrong=1 ewrong=1 total: 900 open/close in 7.74 seconds: 116.28 ops/second llite.lustre-ffff958a09559000.statahead_max=0 1001 real 0m15.501s user 0m0.028s sys 0m2.724s 'ls -l' 1000 files without statahead: 16 sec llite.lustre-ffff958a09559000.statahead_max=32 llite.lustre-ffff958a09559000.statahead_max=32 1001 real 0m5.639s user 0m0.022s sys 0m1.306s 'ls -l' 1000 files with statahead: 5 sec statahead total: 82 statahead wrong: 1 agl total: 77 list_total: 31 fname_total: 50 hit_total: 14902 miss_total: 247 swrong=1 ewrong=1 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 11 seconds llite.lustre-ffff958a09559000.statahead_stats= statahead total: 82 statahead wrong: 1 agl total: 77 list_total: 31 fname_total: 50 hit_total: 14902 miss_total: 247 mdc.lustre-MDT0000-mdc-ffff958a09559000.batch_stats= snapshot_time: 1789371131.896130699 secs.nsecs start_time: 1789371079.467026407 secs.nsecs elapsed_time: 52.429104292 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 1 2 5 4: 2 5 10 8: 2 5 15 16: 18 46 61 32: 15 38 100 mdc.lustre-MDT0001-mdc-ffff958a09559000.batch_stats= snapshot_time: 1789371131.896188305 secs.nsecs start_time: 1789371079.467119907 secs.nsecs elapsed_time: 52.429068398 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 3 7 7 8: 2 5 13 16: 21 55 68 32: 12 31 100 - open/close 1309 (time 1789371143.08 total 10.01 last 130.79) - open/close 2619 (time 1789371153.09 total 20.02 last 130.83) - open/close 3864 (time 1789371163.09 total 30.03 last 124.45) - open/close 5080 (time 1789371173.10 total 40.03 last 121.53) - open/close 6252 (time 1789371183.11 total 50.04 last 117.09) - open/close 7486 (time 1789371193.12 total 60.05 last 123.29) - open/close 8575 (time 1789371203.12 total 70.06 last 108.83) - open/close 9955 (time 1789371213.13 total 80.06 last 137.97) total: 10000 open/close in 80.35 seconds: 124.46 ops/second llite.lustre-ffff958a09559000.statahead_batch_max=0 llite.lustre-ffff958a09559000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff958a09559000.stats=clear mdc.lustre-MDT0001-mdc-ffff958a09559000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m35.449s user 0m0.263s sys 0m19.264s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff958a09559000.statahead_batch_max=32 llite.lustre-ffff958a09559000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff958a09559000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff958a09559000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff958a09559000.stats=clear mdc.lustre-MDT0001-mdc-ffff958a09559000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m41.152s user 0m0.206s sys 0m16.088s batched statahead RPC= hit count=9985 mdc.lustre-MDT0000-mdc-ffff958a09559000.batch_stats= snapshot_time: 1789371298.797379959 secs.nsecs start_time: 1789371252.822676849 secs.nsecs elapsed_time: 45.974703110 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff958a09559000.batch_stats= snapshot_time: 1789371298.797445046 secs.nsecs start_time: 1789371252.822756544 secs.nsecs elapsed_time: 45.974688502 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff958a09559000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff958a09559000.batch_stats=0 llite.lustre-ffff958a09559000.statahead_max=2048 llite.lustre-ffff958a09559000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff958a09559000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff958a09559000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.08 seconds: 92.87 ops/second llite.lustre-ffff958a09559000.statahead_max=0 101 real 0m0.914s user 0m0.006s sys 0m0.272s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff958a09559000.statahead_max=2048 llite.lustre-ffff958a09559000.statahead_max=2048 101 real 0m0.399s user 0m0.001s sys 0m0.196s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10084 miss_total: 16 swrong=0 ewrong=0 total: 900 open/close in 3.62 seconds: 248.92 ops/second llite.lustre-ffff958a09559000.statahead_max=0 1001 real 0m38.451s user 0m0.028s sys 0m3.226s 'ls -l' 1000 files without statahead: 38 sec llite.lustre-ffff958a09559000.statahead_max=2048 llite.lustre-ffff958a09559000.statahead_max=2048 1001 real 0m11.782s user 0m0.037s sys 0m1.543s 'ls -l' 1000 files with statahead: 12 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 28 seconds llite.lustre-ffff958a09559000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 mdc.lustre-MDT0000-mdc-ffff958a09559000.batch_stats= snapshot_time: 1789371521.775735273 secs.nsecs start_time: 1789371299.841616851 secs.nsecs elapsed_time: 221.934118422 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 3 42 42 8: 1 14 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 mdc.lustre-MDT0001-mdc-ffff958a09559000.batch_stats= snapshot_time: 1789371521.776114631 secs.nsecs start_time: 1789371299.841764384 secs.nsecs elapsed_time: 221.934350247 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 28 28 8: 2 28 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 - open/close 816 (time 1789371534.29 total 10.01 last 81.54) - open/close 1683 (time 1789371544.30 total 20.01 last 86.65) - open/close 2546 (time 1789371554.31 total 30.02 last 86.21) - open/close 3587 (time 1789371564.31 total 40.02 last 104.10) - open/close 4661 (time 1789371574.32 total 50.03 last 107.30) - open/close 5681 (time 1789371584.32 total 60.03 last 101.99) - open/close 6712 (time 1789371594.33 total 70.04 last 103.05) - open/close 7675 (time 1789371604.33 total 80.04 last 96.26) - open/close 8736 (time 1789371614.34 total 90.05 last 106.04) - open/close 9650 (time 1789371624.34 total 100.05 last 91.36) total: 10000 open/close in 104.06 seconds: 96.09 ops/second llite.lustre-ffff958a09559000.statahead_batch_max=0 llite.lustre-ffff958a09559000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff958a09559000.stats=clear mdc.lustre-MDT0001-mdc-ffff958a09559000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.659s user 0m0.299s sys 0m14.118s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff958a09559000.statahead_batch_max=256 llite.lustre-ffff958a09559000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff958a09559000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff958a09559000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff958a09559000.stats=clear mdc.lustre-MDT0001-mdc-ffff958a09559000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.826s user 0m0.293s sys 0m14.044s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff958a09559000.batch_stats= snapshot_time: 1789371755.139089148 secs.nsecs start_time: 1789371687.629749531 secs.nsecs elapsed_time: 67.509339617 secs.nsecs subreqs per batch batches %% cum %% 1: 189 56 56 2: 40 11 67 4: 29 8 76 8: 13 3 80 16: 9 2 83 32: 13 3 86 64: 7 2 89 128: 3 0 89 256: 34 10 100 mdc.lustre-MDT0001-mdc-ffff958a09559000.batch_stats= snapshot_time: 1789371755.139160820 secs.nsecs start_time: 1789371687.629872608 secs.nsecs elapsed_time: 67.509288212 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff958a09559000.statahead_batch_max=64 llite.lustre-ffff958a09559000.statahead_max=128