== sanity test 123ad: Verify batching statahead works correctly ========================================================== 10:15:41 (1789136141) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.batch_stats=0 llite.lustre-ffff9e2844d3f800.statahead_max=32 llite.lustre-ffff9e2844d3f800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.batch_stats=0 total: 100 open/close in 2.44 seconds: 41.05 ops/second llite.lustre-ffff9e2844d3f800.statahead_max=0 101 real 0m2.338s user 0m0.006s sys 0m0.289s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9e2844d3f800.statahead_max=32 llite.lustre-ffff9e2844d3f800.statahead_max=32 101 real 0m0.942s user 0m0.004s sys 0m0.147s 'ls -l' 100 files with statahead: 1 sec statahead total: 138 statahead wrong: 2 agl total: 132 list_total: 30 fname_total: 79 hit_total: 4381 miss_total: 1285 swrong=2 ewrong=2 - open/close 768 (time 1789136167.72 total 10.00 last 76.78) total: 900 open/close in 11.79 seconds: 76.33 ops/second llite.lustre-ffff9e2844d3f800.statahead_max=0 1001 real 0m21.228s user 0m0.030s sys 0m2.715s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff9e2844d3f800.statahead_max=32 llite.lustre-ffff9e2844d3f800.statahead_max=32 1001 real 0m9.582s user 0m0.021s sys 0m1.375s 'ls -l' 1000 files with statahead: 9 sec statahead total: 139 statahead wrong: 2 agl total: 133 list_total: 31 fname_total: 79 hit_total: 5379 miss_total: 1287 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9e2844d3f800.statahead_stats= statahead total: 139 statahead wrong: 2 agl total: 133 list_total: 31 fname_total: 79 hit_total: 5379 miss_total: 1287 mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.batch_stats= snapshot_time: 1789136233.992299809 secs.nsecs start_time: 1789136145.849537554 secs.nsecs elapsed_time: 88.142762255 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 723 (time 1789136246.43 total 10.01 last 72.21) - open/close 1436 (time 1789136256.44 total 20.02 last 71.26) - open/close 2199 (time 1789136266.46 total 30.04 last 76.12) - open/close 2952 (time 1789136276.48 total 40.06 last 75.20) - open/close 3680 (time 1789136286.48 total 50.06 last 72.78) - open/close 4383 (time 1789136296.48 total 60.06 last 70.28) - open/close 5186 (time 1789136306.49 total 70.07 last 80.27) - open/close 5907 (time 1789136316.50 total 80.08 last 72.01) - open/close 6651 (time 1789136326.51 total 90.09 last 74.32) - open/close 7390 (time 1789136336.51 total 100.09 last 73.88) - open/close 8076 (time 1789136346.53 total 110.11 last 68.47) - open/close 8787 (time 1789136356.54 total 120.11 last 71.07) - open/close 9521 (time 1789136366.55 total 130.12 last 73.32) total: 10000 open/close in 137.12 seconds: 72.93 ops/second llite.lustre-ffff9e2844d3f800.statahead_batch_max=0 llite.lustre-ffff9e2844d3f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m2.224s user 0m0.269s sys 0m14.461s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9e2844d3f800.statahead_batch_max=32 llite.lustre-ffff9e2844d3f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m35.435s user 0m0.274s sys 0m12.962s batched statahead RPC= hit count=9978 mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.batch_stats= snapshot_time: 1789136550.482399837 secs.nsecs start_time: 1789136443.591225336 secs.nsecs elapsed_time: 106.891174501 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.batch_stats=0 llite.lustre-ffff9e2844d3f800.statahead_max=2048 llite.lustre-ffff9e2844d3f800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.batch_stats=0 total: 100 open/close in 2.21 seconds: 45.23 ops/second llite.lustre-ffff9e2844d3f800.statahead_max=0 101 real 0m2.233s user 0m0.003s sys 0m0.302s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9e2844d3f800.statahead_max=2048 llite.lustre-ffff9e2844d3f800.statahead_max=2048 101 real 0m0.692s user 0m0.007s sys 0m0.160s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10077 miss_total: 23 swrong=0 ewrong=0 - open/close 869 (time 1789136843.03 total 10.00 last 86.88) total: 900 open/close in 10.34 seconds: 87.05 ops/second llite.lustre-ffff9e2844d3f800.statahead_max=0 1001 real 0m17.494s user 0m0.035s sys 0m2.765s 'ls -l' 1000 files without statahead: 17 sec llite.lustre-ffff9e2844d3f800.statahead_max=2048 llite.lustre-ffff9e2844d3f800.statahead_max=2048 1001 real 0m5.024s user 0m0.021s sys 0m1.507s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 16 seconds llite.lustre-ffff9e2844d3f800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11076 miss_total: 24 mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.batch_stats= snapshot_time: 1789136889.581137472 secs.nsecs start_time: 1789136552.944540963 secs.nsecs elapsed_time: 336.636596509 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 1 8 66 256: 4 33 100 - open/close 762 (time 1789136901.52 total 10.01 last 76.14) - open/close 1532 (time 1789136911.53 total 20.02 last 76.92) - open/close 2329 (time 1789136921.54 total 30.02 last 79.66) - open/close 3174 (time 1789136931.56 total 40.04 last 84.33) - open/close 3995 (time 1789136941.56 total 50.04 last 82.09) - open/close 4777 (time 1789136951.56 total 60.05 last 78.17) - open/close 5524 (time 1789136961.56 total 70.05 last 74.69) - open/close 6325 (time 1789136971.57 total 80.05 last 80.08) - open/close 7118 (time 1789136981.57 total 90.06 last 79.27) - open/close 7973 (time 1789136991.58 total 100.07 last 85.43) - open/close 8821 (time 1789137001.59 total 110.07 last 84.73) - open/close 9601 (time 1789137011.60 total 120.08 last 77.93) total: 10000 open/close in 123.98 seconds: 80.66 ops/second llite.lustre-ffff9e2844d3f800.statahead_batch_max=0 llite.lustre-ffff9e2844d3f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.642s user 0m0.282s sys 0m15.782s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9e2844d3f800.statahead_batch_max=256 llite.lustre-ffff9e2844d3f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m39.885s user 0m0.265s sys 0m15.823s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9e2844d3f800.batch_stats= snapshot_time: 1789137121.238442416 secs.nsecs start_time: 1789137070.583675045 secs.nsecs elapsed_time: 50.654767371 secs.nsecs subreqs per batch batches %% cum %% 1: 242 60 60 2: 46 11 72 4: 28 7 79 8: 15 3 83 16: 13 3 86 32: 15 3 90 64: 4 1 91 128: 1 0 91 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9e2844d3f800.statahead_batch_max=64 llite.lustre-ffff9e2844d3f800.statahead_max=128