== sanity test 123ad: Verify batching statahead works correctly ========================================================== 11:14:18 (1787843658) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.batch_stats=0 llite.lustre-ffff9aeb300e8800.statahead_max=32 llite.lustre-ffff9aeb300e8800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.batch_stats=0 total: 100 open/close in 4.28 seconds: 23.35 ops/second llite.lustre-ffff9aeb300e8800.statahead_max=0 101 real 0m2.546s user 0m0.007s sys 0m0.307s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9aeb300e8800.statahead_max=32 llite.lustre-ffff9aeb300e8800.statahead_max=32 101 real 0m1.206s user 0m0.004s sys 0m0.169s 'ls -l' 100 files with statahead: 1 sec statahead total: 135 statahead wrong: 2 agl total: 129 list_total: 30 fname_total: 103 hit_total: 4559 miss_total: 1276 swrong=2 ewrong=2 - open/close 673 (time 1787843691.07 total 10.00 last 67.27) total: 900 open/close in 13.92 seconds: 64.66 ops/second llite.lustre-ffff9aeb300e8800.statahead_max=0 1001 real 0m25.884s user 0m0.034s sys 0m3.232s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff9aeb300e8800.statahead_max=32 llite.lustre-ffff9aeb300e8800.statahead_max=32 1001 real 0m10.874s user 0m0.038s sys 0m1.546s 'ls -l' 1000 files with statahead: 11 sec statahead total: 136 statahead wrong: 2 agl total: 130 list_total: 31 fname_total: 103 hit_total: 5556 miss_total: 1279 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 30 seconds llite.lustre-ffff9aeb300e8800.statahead_stats= statahead total: 136 statahead wrong: 2 agl total: 130 list_total: 31 fname_total: 103 hit_total: 5556 miss_total: 1279 mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.batch_stats= snapshot_time: 1787843778.245064570 secs.nsecs start_time: 1787843665.236521013 secs.nsecs elapsed_time: 113.008543557 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 1 2 10 32: 34 89 100 - open/close 586 (time 1787843791.32 total 10.00 last 58.58) - open/close 1084 (time 1787843801.33 total 20.01 last 49.75) - open/close 1678 (time 1787843811.34 total 30.02 last 59.36) - open/close 2317 (time 1787843821.35 total 40.04 last 63.81) - open/close 2989 (time 1787843831.35 total 50.04 last 67.20) - open/close 3569 (time 1787843841.36 total 60.04 last 57.95) - open/close 4188 (time 1787843851.38 total 70.06 last 61.81) - open/close 4840 (time 1787843861.38 total 80.06 last 65.19) - open/close 5336 (time 1787843871.39 total 90.07 last 49.53) - open/close 5945 (time 1787843881.40 total 100.08 last 60.86) - open/close 6503 (time 1787843891.40 total 110.08 last 55.78) - open/close 6991 (time 1787843901.42 total 120.10 last 48.71) - open/close 7548 (time 1787843911.45 total 130.14 last 55.52) - open/close 8092 (time 1787843921.48 total 140.16 last 54.28) - open/close 8590 (time 1787843931.50 total 150.18 last 49.70) - open/close 9160 (time 1787843941.51 total 160.19 last 56.96) - open/close 9713 (time 1787843951.51 total 170.19 last 55.27) total: 10000 open/close in 174.67 seconds: 57.25 ops/second llite.lustre-ffff9aeb300e8800.statahead_batch_max=0 llite.lustre-ffff9aeb300e8800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m17.092s user 0m0.355s sys 0m15.030s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9aeb300e8800.statahead_batch_max=32 llite.lustre-ffff9aeb300e8800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m52.435s user 0m0.344s sys 0m14.164s batched statahead RPC= hit count=9975 mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.batch_stats= snapshot_time: 1787844168.466212902 secs.nsecs start_time: 1787844043.545611356 secs.nsecs elapsed_time: 124.920601546 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 1 32: 313 98 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.batch_stats=0 llite.lustre-ffff9aeb300e8800.statahead_max=2048 llite.lustre-ffff9aeb300e8800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.batch_stats=0 total: 100 open/close in 3.97 seconds: 25.21 ops/second llite.lustre-ffff9aeb300e8800.statahead_max=0 101 real 0m2.682s user 0m0.003s sys 0m0.379s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9aeb300e8800.statahead_max=2048 llite.lustre-ffff9aeb300e8800.statahead_max=2048 101 real 0m0.928s user 0m0.003s sys 0m0.156s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10074 miss_total: 26 swrong=0 ewrong=0 - open/close 524 (time 1787844529.41 total 10.01 last 52.34) total: 900 open/close in 17.95 seconds: 50.13 ops/second llite.lustre-ffff9aeb300e8800.statahead_max=0 1001 real 0m28.540s user 0m0.031s sys 0m3.182s 'ls -l' 1000 files without statahead: 29 sec llite.lustre-ffff9aeb300e8800.statahead_max=2048 llite.lustre-ffff9aeb300e8800.statahead_max=2048 1001 real 0m7.242s user 0m0.030s sys 0m1.377s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11073 miss_total: 27 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff9aeb300e8800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11073 miss_total: 27 mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.batch_stats= snapshot_time: 1787844610.612061003 secs.nsecs start_time: 1787844170.743750084 secs.nsecs elapsed_time: 439.868310919 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 627 (time 1787844623.40 total 10.02 last 62.58) - open/close 1181 (time 1787844633.41 total 20.03 last 55.33) - open/close 1830 (time 1787844643.41 total 30.03 last 64.90) - open/close 2468 (time 1787844653.43 total 40.05 last 63.69) - open/close 3086 (time 1787844663.43 total 50.05 last 61.77) - open/close 3682 (time 1787844673.43 total 60.05 last 59.59) - open/close 4340 (time 1787844683.44 total 70.06 last 65.74) - open/close 4965 (time 1787844693.45 total 80.07 last 62.46) - open/close 5608 (time 1787844703.45 total 90.08 last 64.26) - open/close 6094 (time 1787844713.48 total 100.10 last 48.48) - open/close 6700 (time 1787844723.48 total 110.10 last 60.57) - open/close 7284 (time 1787844733.48 total 120.10 last 58.39) - open/close 7886 (time 1787844743.49 total 130.11 last 60.19) - open/close 8489 (time 1787844753.49 total 140.12 last 60.24) - open/close 9143 (time 1787844763.50 total 150.12 last 65.37) - open/close 9774 (time 1787844773.51 total 160.13 last 63.06) total: 10000 open/close in 163.86 seconds: 61.03 ops/second llite.lustre-ffff9aeb300e8800.statahead_batch_max=0 llite.lustre-ffff9aeb300e8800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.130s user 0m0.396s sys 0m15.791s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9aeb300e8800.statahead_batch_max=256 llite.lustre-ffff9aeb300e8800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.343s user 0m0.327s sys 0m15.380s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9aeb300e8800.batch_stats= snapshot_time: 1787844926.835361408 secs.nsecs start_time: 1787844858.325919655 secs.nsecs elapsed_time: 68.509441753 secs.nsecs subreqs per batch batches %% cum %% 1: 104 44 44 2: 22 9 54 4: 25 10 64 8: 17 7 72 16: 14 6 78 32: 9 3 81 64: 7 3 84 128: 0 0 84 256: 35 15 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9aeb300e8800.statahead_batch_max=64 llite.lustre-ffff9aeb300e8800.statahead_max=128