== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:36:48 (1787319408) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8aad876f3000.batch_stats=0 llite.lustre-ffff8aad876f3000.statahead_max=32 llite.lustre-ffff8aad876f3000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8aad876f3000.batch_stats=0 total: 100 open/close in 3.28 seconds: 30.45 ops/second llite.lustre-ffff8aad876f3000.statahead_max=0 101 real 0m2.569s user 0m0.004s sys 0m0.333s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8aad876f3000.statahead_max=32 llite.lustre-ffff8aad876f3000.statahead_max=32 101 real 0m1.356s user 0m0.005s sys 0m0.175s 'ls -l' 100 files with statahead: 1 sec statahead total: 116 statahead wrong: 2 agl total: 110 list_total: 31 fname_total: 70 hit_total: 3812 miss_total: 1603 swrong=2 ewrong=2 - open/close 547 (time 1787319437.51 total 10.04 last 54.48) total: 900 open/close in 15.30 seconds: 58.84 ops/second llite.lustre-ffff8aad876f3000.statahead_max=0 1001 real 0m26.570s user 0m0.038s sys 0m3.164s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff8aad876f3000.statahead_max=32 llite.lustre-ffff8aad876f3000.statahead_max=32 1001 real 0m11.907s user 0m0.038s sys 0m1.531s 'ls -l' 1000 files with statahead: 12 sec statahead total: 117 statahead wrong: 2 agl total: 111 list_total: 32 fname_total: 70 hit_total: 4809 miss_total: 1606 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff8aad876f3000.statahead_stats= statahead total: 117 statahead wrong: 2 agl total: 111 list_total: 32 fname_total: 70 hit_total: 4809 miss_total: 1606 mdc.lustre-MDT0000-mdc-ffff8aad876f3000.batch_stats= snapshot_time: 1787319520.335083271 secs.nsecs start_time: 1787319412.894859816 secs.nsecs elapsed_time: 107.440223455 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 1 2 12 32: 34 87 100 - open/close 730 (time 1787319532.83 total 10.02 last 72.85) - open/close 1395 (time 1787319542.84 total 20.03 last 66.44) - open/close 2118 (time 1787319552.86 total 30.05 last 72.16) - open/close 2828 (time 1787319562.89 total 40.07 last 70.81) - open/close 3441 (time 1787319572.89 total 50.08 last 61.29) - open/close 4149 (time 1787319582.89 total 60.08 last 70.78) - open/close 4808 (time 1787319592.90 total 70.09 last 65.83) - open/close 5513 (time 1787319602.91 total 80.09 last 70.47) - open/close 6204 (time 1787319612.91 total 90.10 last 69.08) - open/close 6775 (time 1787319622.92 total 100.11 last 57.01) - open/close 7358 (time 1787319632.93 total 110.11 last 58.29) - open/close 8024 (time 1787319642.93 total 120.12 last 66.58) - open/close 8718 (time 1787319652.94 total 130.12 last 69.35) - open/close 9446 (time 1787319662.94 total 140.13 last 72.77) total: 10000 open/close in 147.79 seconds: 67.66 ops/second llite.lustre-ffff8aad876f3000.statahead_batch_max=0 llite.lustre-ffff8aad876f3000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8aad876f3000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.521s user 0m0.328s sys 0m15.920s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8aad876f3000.statahead_batch_max=32 llite.lustre-ffff8aad876f3000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8aad876f3000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8aad876f3000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m35.690s user 0m0.332s sys 0m15.389s batched statahead RPC= hit count=9989 mdc.lustre-MDT0000-mdc-ffff8aad876f3000.batch_stats= snapshot_time: 1787319857.698633315 secs.nsecs start_time: 1787319748.253596918 secs.nsecs elapsed_time: 109.445036397 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8aad876f3000.batch_stats=0 llite.lustre-ffff8aad876f3000.statahead_max=2048 llite.lustre-ffff8aad876f3000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8aad876f3000.batch_stats=0 total: 100 open/close in 2.48 seconds: 40.38 ops/second llite.lustre-ffff8aad876f3000.statahead_max=0 101 real 0m2.678s user 0m0.006s sys 0m0.340s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8aad876f3000.statahead_max=2048 llite.lustre-ffff8aad876f3000.statahead_max=2048 101 real 0m0.867s user 0m0.002s sys 0m0.154s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10088 miss_total: 12 swrong=0 ewrong=0 - open/close 711 (time 1787320193.68 total 10.01 last 71.05) total: 900 open/close in 12.72 seconds: 70.73 ops/second llite.lustre-ffff8aad876f3000.statahead_max=0 1001 real 0m27.203s user 0m0.032s sys 0m3.140s 'ls -l' 1000 files without statahead: 28 sec llite.lustre-ffff8aad876f3000.statahead_max=2048 llite.lustre-ffff8aad876f3000.statahead_max=2048 1001 real 0m6.948s user 0m0.024s sys 0m1.666s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11087 miss_total: 13 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff8aad876f3000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11087 miss_total: 13 mdc.lustre-MDT0000-mdc-ffff8aad876f3000.batch_stats= snapshot_time: 1787320269.004906930 secs.nsecs start_time: 1787319860.125806820 secs.nsecs elapsed_time: 408.879100110 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 15 15 16: 2 15 30 32: 3 23 53 64: 2 15 69 128: 0 0 69 256: 4 30 100 - open/close 673 (time 1787320281.59 total 10.01 last 67.21) - open/close 1325 (time 1787320291.59 total 20.02 last 65.15) - open/close 2009 (time 1787320301.60 total 30.03 last 68.34) - open/close 2768 (time 1787320311.60 total 40.03 last 75.89) - open/close 3398 (time 1787320321.61 total 50.03 last 62.99) - open/close 4146 (time 1787320331.61 total 60.04 last 74.77) - open/close 4914 (time 1787320341.61 total 70.04 last 76.77) - open/close 5612 (time 1787320351.62 total 80.05 last 69.78) - open/close 6254 (time 1787320361.64 total 90.06 last 64.08) - open/close 7004 (time 1787320371.64 total 100.07 last 74.94) - open/close 7698 (time 1787320381.65 total 110.08 last 69.35) - open/close 8445 (time 1787320391.66 total 120.09 last 74.61) - open/close 9192 (time 1787320401.67 total 130.10 last 74.62) - open/close 9889 (time 1787320411.68 total 140.10 last 69.68) total: 10000 open/close in 141.56 seconds: 70.64 ops/second llite.lustre-ffff8aad876f3000.statahead_batch_max=0 llite.lustre-ffff8aad876f3000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8aad876f3000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.416s user 0m0.335s sys 0m16.818s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8aad876f3000.statahead_batch_max=256 llite.lustre-ffff8aad876f3000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8aad876f3000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8aad876f3000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m3.896s user 0m0.298s sys 0m15.909s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8aad876f3000.batch_stats= snapshot_time: 1787320556.854597748 secs.nsecs start_time: 1787320478.493868961 secs.nsecs elapsed_time: 78.360728787 secs.nsecs subreqs per batch batches %% cum %% 1: 323 52 52 2: 98 15 68 4: 84 13 82 8: 37 6 88 16: 23 3 91 32: 15 2 94 64: 2 0 94 128: 1 0 94 256: 32 5 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8aad876f3000.statahead_batch_max=64 llite.lustre-ffff8aad876f3000.statahead_max=128