== sanity test 123ad: Verify batching statahead works correctly ========================================================== 02:27:11 (1789021631) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff945611887800.batch_stats=0 llite.lustre-ffff945611887800.statahead_max=32 llite.lustre-ffff945611887800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff945611887800.batch_stats=0 total: 100 open/close in 2.85 seconds: 35.14 ops/second llite.lustre-ffff945611887800.statahead_max=0 101 real 0m2.905s user 0m0.005s sys 0m0.310s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff945611887800.statahead_max=32 llite.lustre-ffff945611887800.statahead_max=32 101 real 0m0.866s user 0m0.004s sys 0m0.140s 'ls -l' 100 files with statahead: 1 sec statahead total: 139 statahead wrong: 2 agl total: 133 list_total: 29 fname_total: 83 hit_total: 3909 miss_total: 1281 swrong=2 ewrong=2 - open/close 717 (time 1789021661.79 total 10.01 last 71.66) total: 900 open/close in 12.19 seconds: 73.81 ops/second llite.lustre-ffff945611887800.statahead_max=0 1001 real 0m18.940s user 0m0.030s sys 0m2.753s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff945611887800.statahead_max=32 llite.lustre-ffff945611887800.statahead_max=32 1001 real 0m12.538s user 0m0.035s sys 0m1.393s 'ls -l' 1000 files with statahead: 12 sec statahead total: 140 statahead wrong: 2 agl total: 134 list_total: 30 fname_total: 83 hit_total: 4904 miss_total: 1286 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff945611887800.statahead_stats= statahead total: 140 statahead wrong: 2 agl total: 134 list_total: 30 fname_total: 83 hit_total: 4904 miss_total: 1286 mdc.lustre-MDT0000-mdc-ffff945611887800.batch_stats= snapshot_time: 1789021734.693110431 secs.nsecs start_time: 1789021637.905472596 secs.nsecs elapsed_time: 96.787637835 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 2 5 7 16: 3 7 15 32: 34 85 100 - open/close 622 (time 1789021747.96 total 10.01 last 62.11) - open/close 1386 (time 1789021757.96 total 20.02 last 76.37) - open/close 2120 (time 1789021767.96 total 30.02 last 73.40) - open/close 2877 (time 1789021777.97 total 40.02 last 75.65) - open/close 3692 (time 1789021787.97 total 50.03 last 81.48) - open/close 4452 (time 1789021797.98 total 60.04 last 75.91) - open/close 5099 (time 1789021807.99 total 70.05 last 64.63) - open/close 5707 (time 1789021818.00 total 80.06 last 60.73) - open/close 6431 (time 1789021828.01 total 90.07 last 72.34) - open/close 7221 (time 1789021838.02 total 100.07 last 78.96) - open/close 8167 (time 1789021848.02 total 110.07 last 94.60) - open/close 8887 (time 1789021858.03 total 120.08 last 71.93) - open/close 9620 (time 1789021868.03 total 130.09 last 73.24) total: 10000 open/close in 135.30 seconds: 73.91 ops/second llite.lustre-ffff945611887800.statahead_batch_max=0 llite.lustre-ffff945611887800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff945611887800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.155s user 0m0.313s sys 0m13.357s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff945611887800.statahead_batch_max=32 llite.lustre-ffff945611887800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff945611887800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff945611887800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m39.422s user 0m0.284s sys 0m13.118s batched statahead RPC= hit count=9975 mdc.lustre-MDT0000-mdc-ffff945611887800.batch_stats= snapshot_time: 1789022053.836261281 secs.nsecs start_time: 1789021943.705238090 secs.nsecs elapsed_time: 110.131023191 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff945611887800.batch_stats=0 llite.lustre-ffff945611887800.statahead_max=2048 llite.lustre-ffff945611887800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff945611887800.batch_stats=0 total: 100 open/close in 2.48 seconds: 40.28 ops/second llite.lustre-ffff945611887800.statahead_max=0 101 real 0m2.379s user 0m0.004s sys 0m0.300s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff945611887800.statahead_max=2048 llite.lustre-ffff945611887800.statahead_max=2048 101 real 0m1.011s user 0m0.008s sys 0m0.155s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10074 miss_total: 26 swrong=0 ewrong=0 - open/close 797 (time 1789022355.86 total 10.00 last 79.70) total: 900 open/close in 11.60 seconds: 77.62 ops/second llite.lustre-ffff945611887800.statahead_max=0 1001 real 0m17.898s user 0m0.035s sys 0m2.736s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff945611887800.statahead_max=2048 llite.lustre-ffff945611887800.statahead_max=2048 1001 real 0m8.216s user 0m0.025s sys 0m1.279s 'ls -l' 1000 files with statahead: 9 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11073 miss_total: 27 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff945611887800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11073 miss_total: 27 mdc.lustre-MDT0000-mdc-ffff945611887800.batch_stats= snapshot_time: 1789022422.313382983 secs.nsecs start_time: 1789022056.384643460 secs.nsecs elapsed_time: 365.928739523 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 2 22 44 32: 0 0 44 64: 0 0 44 128: 1 11 55 256: 4 44 100 - open/close 872 (time 1789022435.21 total 10.00 last 87.18) - open/close 1603 (time 1789022445.22 total 20.01 last 73.05) - open/close 2219 (time 1789022455.23 total 30.02 last 61.55) - open/close 2745 (time 1789022465.25 total 40.04 last 52.50) - open/close 3455 (time 1789022475.26 total 50.05 last 70.94) - open/close 4086 (time 1789022485.28 total 60.07 last 62.94) - open/close 4711 (time 1789022495.29 total 70.08 last 62.46) - open/close 5363 (time 1789022505.30 total 80.09 last 65.13) - open/close 6195 (time 1789022515.31 total 90.10 last 83.12) - open/close 6920 (time 1789022525.35 total 100.14 last 72.17) - open/close 7550 (time 1789022535.36 total 110.15 last 62.99) - open/close 8154 (time 1789022545.37 total 120.15 last 60.34) - open/close 8739 (time 1789022555.38 total 130.17 last 58.42) - open/close 9429 (time 1789022565.39 total 140.18 last 68.90) total: 10000 open/close in 146.69 seconds: 68.17 ops/second llite.lustre-ffff945611887800.statahead_batch_max=0 llite.lustre-ffff945611887800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff945611887800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.194s user 0m0.326s sys 0m13.771s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff945611887800.statahead_batch_max=256 llite.lustre-ffff945611887800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff945611887800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff945611887800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.572s user 0m0.309s sys 0m13.428s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff945611887800.batch_stats= snapshot_time: 1789022696.511292510 secs.nsecs start_time: 1789022635.591358476 secs.nsecs elapsed_time: 60.919934034 secs.nsecs subreqs per batch batches %% cum %% 1: 78 40 40 2: 22 11 52 4: 16 8 60 8: 9 4 65 16: 11 5 70 32: 17 8 79 64: 2 1 80 128: 2 1 81 256: 35 18 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff945611887800.statahead_batch_max=64 llite.lustre-ffff945611887800.statahead_max=128