== sanity test 123ad: Verify batching statahead works correctly ========================================================== 02:44:06 (1784702646) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.batch_stats=0 llite.lustre-ffff91dc18fcc000.statahead_max=32 llite.lustre-ffff91dc18fcc000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.batch_stats=0 total: 100 open/close in 1.90 seconds: 52.66 ops/second llite.lustre-ffff91dc18fcc000.statahead_max=0 101 real 0m1.664s user 0m0.004s sys 0m0.306s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff91dc18fcc000.statahead_max=32 llite.lustre-ffff91dc18fcc000.statahead_max=32 101 real 0m0.679s user 0m0.004s sys 0m0.158s 'ls -l' 100 files with statahead: 0 sec statahead total: 100 statahead wrong: 2 agl total: 96 list_total: 29 fname_total: 71 hit_total: 4380 miss_total: 1273 swrong=2 ewrong=2 total: 900 open/close in 7.22 seconds: 124.62 ops/second llite.lustre-ffff91dc18fcc000.statahead_max=0 1001 real 0m14.339s user 0m0.026s sys 0m2.810s 'ls -l' 1000 files without statahead: 14 sec llite.lustre-ffff91dc18fcc000.statahead_max=32 llite.lustre-ffff91dc18fcc000.statahead_max=32 1001 real 0m6.644s user 0m0.023s sys 0m1.514s 'ls -l' 1000 files with statahead: 6 sec statahead total: 101 statahead wrong: 2 agl total: 97 list_total: 30 fname_total: 71 hit_total: 5376 miss_total: 1277 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff91dc18fcc000.statahead_stats= statahead total: 101 statahead wrong: 2 agl total: 97 list_total: 30 fname_total: 71 hit_total: 5376 miss_total: 1277 mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.batch_stats= snapshot_time: 1784702718.150491334 secs.nsecs start_time: 1784702650.185087926 secs.nsecs elapsed_time: 67.965403408 secs.nsecs subreqs per batch batches %% cum %% 1: 2 4 4 2: 0 0 4 4: 0 0 4 8: 3 7 12 16: 2 4 17 32: 34 82 100 - open/close 1205 (time 1784702729.93 total 10.00 last 120.46) - open/close 2398 (time 1784702739.94 total 20.01 last 119.23) - open/close 3591 (time 1784702749.94 total 30.01 last 119.30) - open/close 4685 (time 1784702759.94 total 40.01 last 109.40) - open/close 5702 (time 1784702769.95 total 50.02 last 101.63) - open/close 6763 (time 1784702779.95 total 60.02 last 106.05) - open/close 7729 (time 1784702789.96 total 70.03 last 96.53) - open/close 8858 (time 1784702799.96 total 80.03 last 112.86) - open/close 9936 (time 1784702809.96 total 90.03 last 107.79) total: 10000 open/close in 90.53 seconds: 110.46 ops/second llite.lustre-ffff91dc18fcc000.statahead_batch_max=0 llite.lustre-ffff91dc18fcc000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m39.791s user 0m0.250s sys 0m17.812s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff91dc18fcc000.statahead_batch_max=32 llite.lustre-ffff91dc18fcc000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m38.659s user 0m0.316s sys 0m14.609s batched statahead RPC= hit count=9984 mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.batch_stats= snapshot_time: 1784702964.411299021 secs.nsecs start_time: 1784702856.744612992 secs.nsecs elapsed_time: 107.666686029 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.batch_stats=0 llite.lustre-ffff91dc18fcc000.statahead_max=2048 llite.lustre-ffff91dc18fcc000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.batch_stats=0 total: 100 open/close in 2.07 seconds: 48.27 ops/second llite.lustre-ffff91dc18fcc000.statahead_max=0 101 real 0m1.805s user 0m0.007s sys 0m0.329s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff91dc18fcc000.statahead_max=2048 llite.lustre-ffff91dc18fcc000.statahead_max=2048 101 real 0m0.632s user 0m0.004s sys 0m0.152s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10083 miss_total: 17 swrong=0 ewrong=0 total: 900 open/close in 8.82 seconds: 102.07 ops/second llite.lustre-ffff91dc18fcc000.statahead_max=0 1001 real 0m15.916s user 0m0.029s sys 0m2.881s 'ls -l' 1000 files without statahead: 16 sec llite.lustre-ffff91dc18fcc000.statahead_max=2048 llite.lustre-ffff91dc18fcc000.statahead_max=2048 1001 real 0m5.369s user 0m0.021s sys 0m1.546s 'ls -l' 1000 files with statahead: 5 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 15 seconds llite.lustre-ffff91dc18fcc000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11082 miss_total: 18 mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.batch_stats= snapshot_time: 1784703273.811070480 secs.nsecs start_time: 1784702966.804907634 secs.nsecs elapsed_time: 307.006162846 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 3 25 58 64: 1 8 66 128: 0 0 66 256: 4 33 100 - open/close 989 (time 1784703285.95 total 10.00 last 98.88) - open/close 2048 (time 1784703295.95 total 20.01 last 105.86) - open/close 3136 (time 1784703305.96 total 30.01 last 108.70) - open/close 4261 (time 1784703315.96 total 40.02 last 112.48) - open/close 5268 (time 1784703325.96 total 50.02 last 100.70) - open/close 6410 (time 1784703335.97 total 60.03 last 114.09) - open/close 7405 (time 1784703345.98 total 70.03 last 99.46) - open/close 8466 (time 1784703355.99 total 80.04 last 105.96) - open/close 9567 (time 1784703366.00 total 90.05 last 110.05) total: 10000 open/close in 93.81 seconds: 106.60 ops/second llite.lustre-ffff91dc18fcc000.statahead_batch_max=0 llite.lustre-ffff91dc18fcc000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m43.546s user 0m0.302s sys 0m18.027s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff91dc18fcc000.statahead_batch_max=256 llite.lustre-ffff91dc18fcc000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m42.668s user 0m0.318s sys 0m16.540s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff91dc18fcc000.batch_stats= snapshot_time: 1784703473.190634585 secs.nsecs start_time: 1784703419.926135418 secs.nsecs elapsed_time: 53.264499167 secs.nsecs subreqs per batch batches %% cum %% 1: 325 58 58 2: 75 13 71 4: 63 11 83 8: 26 4 87 16: 19 3 91 32: 15 2 93 64: 0 0 93 128: 0 0 93 256: 34 6 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff91dc18fcc000.statahead_batch_max=64 llite.lustre-ffff91dc18fcc000.statahead_max=128