== sanity test 123ad: Verify batching statahead works correctly ========================================================== 11:05:00 (1788879900) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.batch_stats=0 llite.lustre-ffff8e9590a3b800.statahead_max=32 llite.lustre-ffff8e9590a3b800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.69 seconds: 145.35 ops/second llite.lustre-ffff8e9590a3b800.statahead_max=0 101 real 0m0.647s user 0m0.007s sys 0m0.235s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff8e9590a3b800.statahead_max=32 llite.lustre-ffff8e9590a3b800.statahead_max=32 101 real 0m0.398s user 0m0.004s sys 0m0.196s 'ls -l' 100 files with statahead: 1 sec statahead total: 121 statahead wrong: 2 agl total: 114 list_total: 31 fname_total: 88 hit_total: 14066 miss_total: 1288 swrong=2 ewrong=2 total: 900 open/close in 1.78 seconds: 504.92 ops/second llite.lustre-ffff8e9590a3b800.statahead_max=0 1001 real 0m5.650s user 0m0.020s sys 0m2.049s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff8e9590a3b800.statahead_max=32 llite.lustre-ffff8e9590a3b800.statahead_max=32 1001 real 0m3.428s user 0m0.017s sys 0m1.500s 'ls -l' 1000 files with statahead: 4 sec statahead total: 122 statahead wrong: 2 agl total: 115 list_total: 32 fname_total: 88 hit_total: 15065 miss_total: 1289 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 4 seconds llite.lustre-ffff8e9590a3b800.statahead_stats= statahead total: 122 statahead wrong: 2 agl total: 115 list_total: 32 fname_total: 88 hit_total: 15065 miss_total: 1289 mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.batch_stats= snapshot_time: 1788879923.704227162 secs.nsecs start_time: 1788879901.603388205 secs.nsecs elapsed_time: 22.100838957 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 4 10 12 8: 1 2 15 16: 20 51 66 32: 13 33 100 mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.batch_stats= snapshot_time: 1788879923.704305542 secs.nsecs start_time: 1788879901.603442742 secs.nsecs elapsed_time: 22.100862800 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 5 12 15 16: 19 48 64 32: 14 35 100 - open/close 4887 (time 1788879934.98 total 10.00 last 488.62) - open/close 9946 (time 1788879944.98 total 20.00 last 505.86) total: 10000 open/close in 20.10 seconds: 497.41 ops/second llite.lustre-ffff8e9590a3b800.statahead_batch_max=0 llite.lustre-ffff8e9590a3b800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.stats=clear mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.257s user 0m0.159s sys 0m29.119s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8e9590a3b800.statahead_batch_max=32 llite.lustre-ffff8e9590a3b800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.stats=clear mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m33.022s user 0m0.166s sys 0m19.183s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.batch_stats= snapshot_time: 1788880014.834848972 secs.nsecs start_time: 1788879977.761052981 secs.nsecs elapsed_time: 37.073795991 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.batch_stats= snapshot_time: 1788880014.834903715 secs.nsecs start_time: 1788879977.761101247 secs.nsecs elapsed_time: 37.073802468 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.batch_stats=0 llite.lustre-ffff8e9590a3b800.statahead_max=2048 llite.lustre-ffff8e9590a3b800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.81 seconds: 123.91 ops/second llite.lustre-ffff8e9590a3b800.statahead_max=0 101 real 0m0.655s user 0m0.004s sys 0m0.240s 'ls -l' 100 files without statahead: 0 sec llite.lustre-ffff8e9590a3b800.statahead_max=2048 llite.lustre-ffff8e9590a3b800.statahead_max=2048 101 real 0m0.390s user 0m0.003s sys 0m0.228s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 swrong=0 ewrong=0 total: 900 open/close in 2.02 seconds: 446.62 ops/second llite.lustre-ffff8e9590a3b800.statahead_max=0 1001 real 0m5.911s user 0m0.018s sys 0m2.135s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff8e9590a3b800.statahead_max=2048 llite.lustre-ffff8e9590a3b800.statahead_max=2048 1001 real 0m3.180s user 0m0.014s sys 0m1.353s 'ls -l' 1000 files with statahead: 3 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 4 seconds llite.lustre-ffff8e9590a3b800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.batch_stats= snapshot_time: 1788880121.076432989 secs.nsecs start_time: 1788880015.562595290 secs.nsecs elapsed_time: 105.513837699 secs.nsecs subreqs per batch batches %% cum %% 1: 2 25 25 2: 0 0 25 4: 3 37 62 8: 1 12 75 16: 0 0 75 32: 0 0 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.batch_stats= snapshot_time: 1788880121.076531787 secs.nsecs start_time: 1788880015.562729363 secs.nsecs elapsed_time: 105.513802424 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 11 11 8: 3 33 44 16: 1 11 55 32: 1 11 66 64: 0 0 66 128: 1 11 77 256: 2 22 100 - open/close 4830 (time 1788880132.96 total 10.00 last 482.94) - open/close 9645 (time 1788880142.96 total 20.00 last 481.48) total: 10000 open/close in 20.76 seconds: 481.61 ops/second llite.lustre-ffff8e9590a3b800.statahead_batch_max=0 llite.lustre-ffff8e9590a3b800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.stats=clear mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m30.646s user 0m0.157s sys 0m26.488s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8e9590a3b800.statahead_batch_max=256 llite.lustre-ffff8e9590a3b800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.stats=clear mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m25.611s user 0m0.144s sys 0m19.348s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8e9590a3b800.batch_stats= snapshot_time: 1788880206.845770827 secs.nsecs start_time: 1788880176.845540276 secs.nsecs elapsed_time: 30.000230551 secs.nsecs subreqs per batch batches %% cum %% 1: 293 61 61 2: 62 13 74 4: 42 8 83 8: 17 3 87 16: 1 0 87 32: 18 3 91 64: 7 1 92 128: 0 0 92 256: 34 7 100 mdc.lustre-MDT0001-mdc-ffff8e9590a3b800.batch_stats= snapshot_time: 1788880206.846110848 secs.nsecs start_time: 1788880176.845594143 secs.nsecs elapsed_time: 30.000516705 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8e9590a3b800.statahead_batch_max=64 llite.lustre-ffff8e9590a3b800.statahead_max=128