== sanity test 123ad: Verify batching statahead works correctly ========================================================== 23:42:02 (1789443722) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff95c820172800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff95c820172800.batch_stats=0 llite.lustre-ffff95c820172800.statahead_max=32 llite.lustre-ffff95c820172800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff95c820172800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff95c820172800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.71 seconds: 36.87 ops/second llite.lustre-ffff95c820172800.statahead_max=0 101 real 0m2.619s user 0m0.002s sys 0m0.304s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff95c820172800.statahead_max=32 llite.lustre-ffff95c820172800.statahead_max=32 101 real 0m1.061s user 0m0.005s sys 0m0.186s 'ls -l' 100 files with statahead: 1 sec statahead total: 147 statahead wrong: 2 agl total: 141 list_total: 43 fname_total: 104 hit_total: 3738 miss_total: 1291 swrong=2 ewrong=2 - open/close 707 (time 1789443749.02 total 10.01 last 70.60) total: 900 open/close in 13.19 seconds: 68.25 ops/second llite.lustre-ffff95c820172800.statahead_max=0 1001 real 0m24.861s user 0m0.038s sys 0m3.073s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff95c820172800.statahead_max=32 llite.lustre-ffff95c820172800.statahead_max=32 1001 real 0m8.315s user 0m0.032s sys 0m1.417s 'ls -l' 1000 files with statahead: 8 sec statahead total: 148 statahead wrong: 2 agl total: 142 list_total: 44 fname_total: 104 hit_total: 4737 miss_total: 1292 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff95c820172800.statahead_stats= statahead total: 148 statahead wrong: 2 agl total: 142 list_total: 44 fname_total: 104 hit_total: 4737 miss_total: 1292 mdc.lustre-MDT0000-mdc-ffff95c820172800.batch_stats= snapshot_time: 1789443818.102875583 secs.nsecs start_time: 1789443726.483355666 secs.nsecs elapsed_time: 91.619519917 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 1 2 5 8: 5 12 17 16: 18 46 64 32: 14 35 100 mdc.lustre-MDT0001-mdc-ffff95c820172800.batch_stats= snapshot_time: 1789443818.103040681 secs.nsecs start_time: 1789443726.483434565 secs.nsecs elapsed_time: 91.619606116 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 2 5 7 4: 2 5 12 8: 1 2 15 16: 22 55 70 32: 12 30 100 - open/close 700 (time 1789443830.53 total 10.01 last 69.95) - open/close 1443 (time 1789443840.54 total 20.02 last 74.22) - open/close 2172 (time 1789443850.55 total 30.03 last 72.84) - open/close 2914 (time 1789443860.55 total 40.03 last 74.18) - open/close 3644 (time 1789443870.58 total 50.05 last 72.81) - open/close 4363 (time 1789443880.60 total 60.08 last 71.74) - open/close 5095 (time 1789443890.60 total 70.08 last 73.16) - open/close 5814 (time 1789443900.61 total 80.09 last 71.84) - open/close 6522 (time 1789443910.62 total 90.10 last 70.76) - open/close 7246 (time 1789443920.63 total 100.11 last 72.32) - open/close 8011 (time 1789443930.64 total 110.12 last 76.42) - open/close 8699 (time 1789443940.64 total 120.12 last 68.78) - open/close 9436 (time 1789443950.67 total 130.15 last 73.51) total: 10000 open/close in 139.40 seconds: 71.74 ops/second llite.lustre-ffff95c820172800.statahead_batch_max=0 llite.lustre-ffff95c820172800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95c820172800.stats=clear mdc.lustre-MDT0001-mdc-ffff95c820172800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.942s user 0m0.328s sys 0m17.332s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95c820172800.statahead_batch_max=32 llite.lustre-ffff95c820172800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95c820172800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff95c820172800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95c820172800.stats=clear mdc.lustre-MDT0001-mdc-ffff95c820172800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m35.602s user 0m0.362s sys 0m15.769s batched statahead RPC= hit count=9981 mdc.lustre-MDT0000-mdc-ffff95c820172800.batch_stats= snapshot_time: 1789444138.129556195 secs.nsecs start_time: 1789444032.971001890 secs.nsecs elapsed_time: 105.158554305 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff95c820172800.batch_stats= snapshot_time: 1789444138.129998304 secs.nsecs start_time: 1789444032.971096393 secs.nsecs elapsed_time: 105.158901911 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff95c820172800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff95c820172800.batch_stats=0 llite.lustre-ffff95c820172800.statahead_max=2048 llite.lustre-ffff95c820172800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff95c820172800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff95c820172800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 3.19 seconds: 31.32 ops/second llite.lustre-ffff95c820172800.statahead_max=0 101 real 0m3.308s user 0m0.004s sys 0m0.356s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff95c820172800.statahead_max=2048 llite.lustre-ffff95c820172800.statahead_max=2048 101 real 0m1.096s user 0m0.003s sys 0m0.172s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10080 miss_total: 20 swrong=0 ewrong=0 - open/close 636 (time 1789444452.71 total 10.01 last 63.54) total: 900 open/close in 13.56 seconds: 66.38 ops/second llite.lustre-ffff95c820172800.statahead_max=0 1001 real 0m24.765s user 0m0.042s sys 0m3.241s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff95c820172800.statahead_max=2048 llite.lustre-ffff95c820172800.statahead_max=2048 1001 real 0m8.522s user 0m0.029s sys 0m1.384s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff95c820172800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 mdc.lustre-MDT0000-mdc-ffff95c820172800.batch_stats= snapshot_time: 1789444527.051454735 secs.nsecs start_time: 1789444140.717704043 secs.nsecs elapsed_time: 386.333750692 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 4 50 50 8: 0 0 50 16: 0 0 50 32: 1 12 62 64: 1 12 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff95c820172800.batch_stats= snapshot_time: 1789444527.051517185 secs.nsecs start_time: 1789444140.718484889 secs.nsecs elapsed_time: 386.333032296 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 25 25 8: 2 25 50 16: 0 0 50 32: 1 12 62 64: 1 12 75 128: 0 0 75 256: 2 25 100 - open/close 608 (time 1789444541.13 total 10.01 last 60.74) - open/close 1326 (time 1789444551.14 total 20.01 last 71.77) - open/close 1991 (time 1789444561.15 total 30.03 last 66.42) - open/close 2750 (time 1789444571.15 total 40.03 last 75.89) - open/close 3602 (time 1789444581.16 total 50.04 last 85.13) - open/close 4366 (time 1789444591.16 total 60.04 last 76.35) - open/close 5096 (time 1789444601.17 total 70.05 last 72.97) - open/close 5884 (time 1789444611.18 total 80.06 last 78.71) - open/close 6641 (time 1789444621.19 total 90.07 last 75.59) - open/close 7329 (time 1789444631.21 total 100.09 last 68.68) - open/close 7984 (time 1789444641.23 total 110.11 last 65.38) - open/close 8597 (time 1789444651.23 total 120.11 last 61.28) - open/close 9260 (time 1789444661.24 total 130.12 last 66.25) - open/close 9967 (time 1789444671.25 total 140.13 last 70.62) total: 10000 open/close in 140.70 seconds: 71.07 ops/second llite.lustre-ffff95c820172800.statahead_batch_max=0 llite.lustre-ffff95c820172800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95c820172800.stats=clear mdc.lustre-MDT0001-mdc-ffff95c820172800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.015s user 0m0.406s sys 0m16.872s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95c820172800.statahead_batch_max=256 llite.lustre-ffff95c820172800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95c820172800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff95c820172800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95c820172800.stats=clear mdc.lustre-MDT0001-mdc-ffff95c820172800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m53.805s user 0m0.286s sys 0m15.835s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff95c820172800.batch_stats= snapshot_time: 1789444810.175154756 secs.nsecs start_time: 1789444745.897613901 secs.nsecs elapsed_time: 64.277540855 secs.nsecs subreqs per batch batches %% cum %% 1: 98 44 44 2: 19 8 53 4: 15 6 60 8: 16 7 67 16: 13 5 73 32: 12 5 78 64: 10 4 83 128: 2 0 84 256: 34 15 100 mdc.lustre-MDT0001-mdc-ffff95c820172800.batch_stats= snapshot_time: 1789444810.175429051 secs.nsecs start_time: 1789444745.897712571 secs.nsecs elapsed_time: 64.277716480 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff95c820172800.statahead_batch_max=64 llite.lustre-ffff95c820172800.statahead_max=128