== sanity test 123ad: Verify batching statahead works correctly ========================================================== 12:20:11 (1790785211) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff95ce1042b000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff95ce1042b000.batch_stats=0 llite.lustre-ffff95ce1042b000.statahead_max=32 llite.lustre-ffff95ce1042b000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff95ce1042b000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff95ce1042b000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.65 seconds: 37.71 ops/second llite.lustre-ffff95ce1042b000.statahead_max=0 101 real 0m2.030s user 0m0.002s sys 0m0.307s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff95ce1042b000.statahead_max=32 llite.lustre-ffff95ce1042b000.statahead_max=32 101 real 0m0.993s user 0m0.009s sys 0m0.148s 'ls -l' 100 files with statahead: 1 sec statahead total: 123 statahead wrong: 2 agl total: 117 list_total: 30 fname_total: 87 hit_total: 3873 miss_total: 1133 swrong=2 ewrong=2 - open/close 810 (time 1790785238.35 total 10.01 last 80.94) total: 900 open/close in 11.17 seconds: 80.56 ops/second llite.lustre-ffff95ce1042b000.statahead_max=0 1001 real 0m25.127s user 0m0.040s sys 0m2.970s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff95ce1042b000.statahead_max=32 llite.lustre-ffff95ce1042b000.statahead_max=32 1001 real 0m9.687s user 0m0.028s sys 0m1.372s 'ls -l' 1000 files with statahead: 10 sec statahead total: 124 statahead wrong: 2 agl total: 118 list_total: 31 fname_total: 87 hit_total: 4867 miss_total: 1139 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff95ce1042b000.statahead_stats= statahead total: 124 statahead wrong: 2 agl total: 118 list_total: 31 fname_total: 87 hit_total: 4867 miss_total: 1139 mdc.lustre-MDT0000-mdc-ffff95ce1042b000.batch_stats= snapshot_time: 1790785309.318311526 secs.nsecs start_time: 1790785215.544816194 secs.nsecs elapsed_time: 93.773495332 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 2 5 7 8: 3 7 15 16: 21 53 69 32: 12 30 100 mdc.lustre-MDT0001-mdc-ffff95ce1042b000.batch_stats= snapshot_time: 1790785309.318390307 secs.nsecs start_time: 1790785215.544974533 secs.nsecs elapsed_time: 93.773415774 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 5 5 8: 4 10 15 16: 22 56 71 32: 11 28 100 - open/close 885 (time 1790785322.19 total 10.00 last 88.48) - open/close 1752 (time 1790785332.19 total 20.00 last 86.70) - open/close 2613 (time 1790785342.19 total 30.01 last 86.05) - open/close 3498 (time 1790785352.19 total 40.01 last 88.49) - open/close 4395 (time 1790785362.20 total 50.01 last 89.67) - open/close 5473 (time 1790785372.20 total 60.02 last 107.77) - open/close 6407 (time 1790785382.20 total 70.02 last 93.38) - open/close 7380 (time 1790785392.21 total 80.02 last 97.24) - open/close 8408 (time 1790785402.21 total 90.03 last 102.76) - open/close 9273 (time 1790785412.22 total 100.04 last 86.40) total: 10000 open/close in 107.29 seconds: 93.20 ops/second llite.lustre-ffff95ce1042b000.statahead_batch_max=0 llite.lustre-ffff95ce1042b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95ce1042b000.stats=clear mdc.lustre-MDT0001-mdc-ffff95ce1042b000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.872s user 0m0.301s sys 0m14.088s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95ce1042b000.statahead_batch_max=32 llite.lustre-ffff95ce1042b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95ce1042b000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff95ce1042b000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95ce1042b000.stats=clear mdc.lustre-MDT0001-mdc-ffff95ce1042b000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m41.859s user 0m0.343s sys 0m13.316s batched statahead RPC= hit count=9978 mdc.lustre-MDT0000-mdc-ffff95ce1042b000.batch_stats= snapshot_time: 1790785600.267714611 secs.nsecs start_time: 1790785486.565791268 secs.nsecs elapsed_time: 113.701923343 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff95ce1042b000.batch_stats= snapshot_time: 1790785600.267772662 secs.nsecs start_time: 1790785486.566244770 secs.nsecs elapsed_time: 113.701527892 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff95ce1042b000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff95ce1042b000.batch_stats=0 llite.lustre-ffff95ce1042b000.statahead_max=2048 llite.lustre-ffff95ce1042b000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff95ce1042b000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff95ce1042b000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.51 seconds: 39.91 ops/second llite.lustre-ffff95ce1042b000.statahead_max=0 101 real 0m2.716s user 0m0.004s sys 0m0.314s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff95ce1042b000.statahead_max=2048 llite.lustre-ffff95ce1042b000.statahead_max=2048 101 real 0m1.053s user 0m0.007s sys 0m0.178s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10077 miss_total: 23 swrong=0 ewrong=0 total: 900 open/close in 9.00 seconds: 100.05 ops/second llite.lustre-ffff95ce1042b000.statahead_max=0 1001 real 0m18.534s user 0m0.030s sys 0m2.885s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff95ce1042b000.statahead_max=2048 llite.lustre-ffff95ce1042b000.statahead_max=2048 1001 real 0m5.351s user 0m0.022s sys 0m1.155s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff95ce1042b000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 mdc.lustre-MDT0000-mdc-ffff95ce1042b000.batch_stats= snapshot_time: 1790785942.492750721 secs.nsecs start_time: 1790785603.500753040 secs.nsecs elapsed_time: 338.991997681 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 10 10 4: 2 20 30 8: 2 20 50 16: 1 10 60 32: 2 20 80 64: 0 0 80 128: 0 0 80 256: 2 20 100 mdc.lustre-MDT0001-mdc-ffff95ce1042b000.batch_stats= snapshot_time: 1790785942.492824211 secs.nsecs start_time: 1790785603.500913880 secs.nsecs elapsed_time: 338.991910331 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 10 10 8: 3 30 40 16: 2 20 60 32: 2 20 80 64: 0 0 80 128: 0 0 80 256: 2 20 100 - open/close 872 (time 1790785955.09 total 10.00 last 87.18) - open/close 1784 (time 1790785965.09 total 20.01 last 91.15) - open/close 2678 (time 1790785975.09 total 30.01 last 89.39) - open/close 3410 (time 1790785985.11 total 40.02 last 73.08) - open/close 4180 (time 1790785995.12 total 50.03 last 76.93) - open/close 5042 (time 1790786005.13 total 60.04 last 86.14) - open/close 6014 (time 1790786015.13 total 70.05 last 97.12) - open/close 6863 (time 1790786025.15 total 80.07 last 84.76) - open/close 7645 (time 1790786035.18 total 90.09 last 78.01) - open/close 8463 (time 1790786045.19 total 100.10 last 81.69) - open/close 9338 (time 1790786055.19 total 110.10 last 87.49) total: 10000 open/close in 119.34 seconds: 83.79 ops/second llite.lustre-ffff95ce1042b000.statahead_batch_max=0 llite.lustre-ffff95ce1042b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95ce1042b000.stats=clear mdc.lustre-MDT0001-mdc-ffff95ce1042b000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.881s user 0m0.344s sys 0m14.299s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff95ce1042b000.statahead_batch_max=256 llite.lustre-ffff95ce1042b000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff95ce1042b000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff95ce1042b000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff95ce1042b000.stats=clear mdc.lustre-MDT0001-mdc-ffff95ce1042b000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.139s user 0m0.306s sys 0m13.978s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff95ce1042b000.batch_stats= snapshot_time: 1790786210.695462216 secs.nsecs start_time: 1790786145.489144175 secs.nsecs elapsed_time: 65.206318041 secs.nsecs subreqs per batch batches %% cum %% 1: 140 45 45 2: 53 17 62 4: 31 10 72 8: 19 6 78 16: 8 2 81 32: 17 5 86 64: 5 1 88 128: 2 0 88 256: 34 11 100 mdc.lustre-MDT0001-mdc-ffff95ce1042b000.batch_stats= snapshot_time: 1790786210.697494867 secs.nsecs start_time: 1790786145.489235211 secs.nsecs elapsed_time: 65.208259656 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff95ce1042b000.statahead_batch_max=64 llite.lustre-ffff95ce1042b000.statahead_max=128