== sanity test 123ad: Verify batching statahead works correctly ========================================================== 18:30:51 (1781217051) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8e6b50289000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8e6b50289000.batch_stats=0 llite.lustre-ffff8e6b50289000.statahead_max=32 llite.lustre-ffff8e6b50289000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8e6b50289000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8e6b50289000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.57 seconds: 38.97 ops/second llite.lustre-ffff8e6b50289000.statahead_max=0 101 real 0m2.572s user 0m0.006s sys 0m0.342s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8e6b50289000.statahead_max=32 llite.lustre-ffff8e6b50289000.statahead_max=32 101 real 0m1.004s user 0m0.004s sys 0m0.189s 'ls -l' 100 files with statahead: 1 sec statahead total: 155 statahead wrong: 2 agl total: 149 list_total: 37 fname_total: 110 hit_total: 3592 miss_total: 1486 swrong=2 ewrong=2 - open/close 740 (time 1781217077.55 total 10.01 last 73.95) total: 900 open/close in 12.45 seconds: 72.30 ops/second llite.lustre-ffff8e6b50289000.statahead_max=0 1001 real 0m25.932s user 0m0.041s sys 0m3.655s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff8e6b50289000.statahead_max=32 llite.lustre-ffff8e6b50289000.statahead_max=32 1001 real 0m10.676s user 0m0.033s sys 0m1.628s 'ls -l' 1000 files with statahead: 11 sec statahead total: 156 statahead wrong: 2 agl total: 150 list_total: 38 fname_total: 110 hit_total: 4587 miss_total: 1491 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff8e6b50289000.statahead_stats= statahead total: 156 statahead wrong: 2 agl total: 150 list_total: 38 fname_total: 110 hit_total: 4587 miss_total: 1491 mdc.lustre-MDT0000-mdc-ffff8e6b50289000.batch_stats= snapshot_time: 1781217151.810390043 secs.nsecs start_time: 1781217055.334087825 secs.nsecs elapsed_time: 96.476302218 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 1 2 2 4: 2 5 7 8: 3 7 15 16: 20 51 66 32: 13 33 100 mdc.lustre-MDT0001-mdc-ffff8e6b50289000.batch_stats= snapshot_time: 1781217151.810468891 secs.nsecs start_time: 1781217055.334257393 secs.nsecs elapsed_time: 96.476211498 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 0 0 2 4: 3 7 10 8: 3 7 17 16: 18 45 62 32: 15 37 100 - open/close 772 (time 1781217164.58 total 10.00 last 77.18) - open/close 1440 (time 1781217174.58 total 20.01 last 66.78) - open/close 2119 (time 1781217184.59 total 30.01 last 67.84) - open/close 2774 (time 1781217194.61 total 40.03 last 65.37) - open/close 3402 (time 1781217204.61 total 50.04 last 62.79) - open/close 4046 (time 1781217214.62 total 60.04 last 64.35) - open/close 4761 (time 1781217224.63 total 70.06 last 71.41) - open/close 5457 (time 1781217234.64 total 80.06 last 69.56) - open/close 6114 (time 1781217244.65 total 90.07 last 65.65) - open/close 6859 (time 1781217254.65 total 100.07 last 74.48) - open/close 7536 (time 1781217264.65 total 110.08 last 67.67) - open/close 8302 (time 1781217274.65 total 120.08 last 76.60) - open/close 8958 (time 1781217284.67 total 130.09 last 65.51) - open/close 9625 (time 1781217294.68 total 140.10 last 66.62) total: 10000 open/close in 145.28 seconds: 68.83 ops/second llite.lustre-ffff8e6b50289000.statahead_batch_max=0 llite.lustre-ffff8e6b50289000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8e6b50289000.stats=clear mdc.lustre-MDT0001-mdc-ffff8e6b50289000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m16.412s user 0m0.381s sys 0m17.956s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8e6b50289000.statahead_batch_max=32 llite.lustre-ffff8e6b50289000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8e6b50289000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8e6b50289000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8e6b50289000.stats=clear mdc.lustre-MDT0001-mdc-ffff8e6b50289000.stats=clear ls -l files in d123ad.sanity: 10001 real 2m18.354s user 0m0.395s sys 0m17.096s batched statahead RPC= hit count=9979 mdc.lustre-MDT0000-mdc-ffff8e6b50289000.batch_stats= snapshot_time: 1781217541.934539865 secs.nsecs start_time: 1781217387.675044094 secs.nsecs elapsed_time: 154.259495771 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff8e6b50289000.batch_stats= snapshot_time: 1781217541.934612580 secs.nsecs start_time: 1781217387.675149658 secs.nsecs elapsed_time: 154.259462922 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8e6b50289000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8e6b50289000.batch_stats=0 llite.lustre-ffff8e6b50289000.statahead_max=2048 llite.lustre-ffff8e6b50289000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8e6b50289000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8e6b50289000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 3.26 seconds: 30.64 ops/second llite.lustre-ffff8e6b50289000.statahead_max=0 101 real 0m3.286s user 0m0.008s sys 0m0.415s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8e6b50289000.statahead_max=2048 llite.lustre-ffff8e6b50289000.statahead_max=2048 101 real 0m0.941s user 0m0.011s sys 0m0.195s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10078 miss_total: 22 swrong=0 ewrong=0 - open/close 580 (time 1781217934.44 total 10.01 last 57.94) total: 900 open/close in 15.31 seconds: 58.79 ops/second llite.lustre-ffff8e6b50289000.statahead_max=0 1001 real 0m33.204s user 0m0.049s sys 0m3.706s 'ls -l' 1000 files without statahead: 33 sec llite.lustre-ffff8e6b50289000.statahead_max=2048 llite.lustre-ffff8e6b50289000.statahead_max=2048 1001 real 0m7.709s user 0m0.035s sys 0m1.583s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff8e6b50289000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 mdc.lustre-MDT0000-mdc-ffff8e6b50289000.batch_stats= snapshot_time: 1781218021.897544091 secs.nsecs start_time: 1781217545.271279003 secs.nsecs elapsed_time: 476.626265088 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 3 30 30 8: 2 20 50 16: 2 20 70 32: 1 10 80 64: 0 0 80 128: 0 0 80 256: 2 20 100 mdc.lustre-MDT0001-mdc-ffff8e6b50289000.batch_stats= snapshot_time: 1781218021.897654564 secs.nsecs start_time: 1781217545.271436855 secs.nsecs elapsed_time: 476.626217709 secs.nsecs subreqs per batch batches % cum % 1: 1 10 10 2: 0 0 10 4: 2 20 30 8: 2 20 50 16: 1 10 60 32: 1 10 70 64: 1 10 80 128: 0 0 80 256: 2 20 100 - open/close 513 (time 1781218034.50 total 10.01 last 51.22) - open/close 1080 (time 1781218044.51 total 20.03 last 56.63) - open/close 1588 (time 1781218054.52 total 30.03 last 50.76) - open/close 2157 (time 1781218064.52 total 40.04 last 56.89) - open/close 2750 (time 1781218074.53 total 50.04 last 59.25) - open/close 3327 (time 1781218084.54 total 60.05 last 57.64) - open/close 3908 (time 1781218094.54 total 70.05 last 58.10) - open/close 4553 (time 1781218104.54 total 80.06 last 64.47) - open/close 5158 (time 1781218114.56 total 90.07 last 60.41) - open/close 5715 (time 1781218124.57 total 100.09 last 55.61) - open/close 6357 (time 1781218134.58 total 110.10 last 64.15) - open/close 6948 (time 1781218144.60 total 120.11 last 59.00) - open/close 7518 (time 1781218154.61 total 130.13 last 56.92) - open/close 8081 (time 1781218164.62 total 140.14 last 56.25) - open/close 8652 (time 1781218174.65 total 150.17 last 56.92) - open/close 9215 (time 1781218184.66 total 160.18 last 56.25) - open/close 9870 (time 1781218194.68 total 170.19 last 65.40) total: 10000 open/close in 172.10 seconds: 58.11 ops/second llite.lustre-ffff8e6b50289000.statahead_batch_max=0 llite.lustre-ffff8e6b50289000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8e6b50289000.stats=clear mdc.lustre-MDT0001-mdc-ffff8e6b50289000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m19.022s user 0m0.403s sys 0m18.585s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8e6b50289000.statahead_batch_max=256 llite.lustre-ffff8e6b50289000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8e6b50289000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8e6b50289000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8e6b50289000.stats=clear mdc.lustre-MDT0001-mdc-ffff8e6b50289000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.186s user 0m0.334s sys 0m18.546s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8e6b50289000.batch_stats= snapshot_time: 1781218363.419312750 secs.nsecs start_time: 1781218288.210487813 secs.nsecs elapsed_time: 75.208824937 secs.nsecs subreqs per batch batches % cum % 1: 245 51 51 2: 77 16 68 4: 58 12 80 8: 23 4 85 16: 13 2 88 32: 15 3 91 64: 7 1 92 128: 1 0 93 256: 33 6 100 mdc.lustre-MDT0001-mdc-ffff8e6b50289000.batch_stats= snapshot_time: 1781218363.419391463 secs.nsecs start_time: 1781218288.211675283 secs.nsecs elapsed_time: 75.207716180 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8e6b50289000.statahead_batch_max=64 llite.lustre-ffff8e6b50289000.statahead_max=128