== sanity test 123ad: Verify batching statahead works correctly ========================================================== 04:24:27 (1789460667) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.batch_stats=0 llite.lustre-ffff9d7158fc1800.statahead_max=32 llite.lustre-ffff9d7158fc1800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.batch_stats=0 total: 100 open/close in 2.94 seconds: 33.97 ops/second llite.lustre-ffff9d7158fc1800.statahead_max=0 101 real 0m2.386s user 0m0.006s sys 0m0.356s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9d7158fc1800.statahead_max=32 llite.lustre-ffff9d7158fc1800.statahead_max=32 101 real 0m1.003s user 0m0.006s sys 0m0.173s 'ls -l' 100 files with statahead: 1 sec statahead total: 117 statahead wrong: 1 agl total: 111 list_total: 30 fname_total: 66 hit_total: 3822 miss_total: 259 swrong=1 ewrong=1 - open/close 577 (time 1789460696.92 total 10.02 last 57.57) total: 900 open/close in 13.98 seconds: 64.37 ops/second llite.lustre-ffff9d7158fc1800.statahead_max=0 1001 real 0m23.960s user 0m0.038s sys 0m2.958s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff9d7158fc1800.statahead_max=32 llite.lustre-ffff9d7158fc1800.statahead_max=32 1001 real 0m12.875s user 0m0.032s sys 0m1.336s 'ls -l' 1000 files with statahead: 13 sec statahead total: 118 statahead wrong: 1 agl total: 112 list_total: 31 fname_total: 66 hit_total: 4819 miss_total: 262 swrong=1 ewrong=1 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9d7158fc1800.statahead_stats= statahead total: 118 statahead wrong: 1 agl total: 112 list_total: 31 fname_total: 66 hit_total: 4819 miss_total: 262 mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.batch_stats= snapshot_time: 1789460773.183651839 secs.nsecs start_time: 1789460672.992313914 secs.nsecs elapsed_time: 100.191337925 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 641 (time 1789460786.05 total 10.01 last 64.05) - open/close 1208 (time 1789460796.06 total 20.02 last 56.66) - open/close 1792 (time 1789460806.08 total 30.03 last 58.29) - open/close 2397 (time 1789460816.09 total 40.05 last 60.43) - open/close 3075 (time 1789460826.10 total 50.05 last 67.76) - open/close 3697 (time 1789460836.10 total 60.06 last 62.18) - open/close 4376 (time 1789460846.10 total 70.06 last 67.89) - open/close 4995 (time 1789460856.12 total 80.07 last 61.82) - open/close 5621 (time 1789460866.15 total 90.10 last 62.40) - open/close 6270 (time 1789460876.15 total 100.10 last 64.89) - open/close 6948 (time 1789460886.16 total 110.11 last 67.75) - open/close 7561 (time 1789460896.16 total 120.12 last 61.26) - open/close 8103 (time 1789460906.17 total 130.13 last 54.15) - open/close 8832 (time 1789460916.18 total 140.14 last 72.80) - open/close 9454 (time 1789460926.19 total 150.14 last 62.19) total: 10000 open/close in 158.88 seconds: 62.94 ops/second llite.lustre-ffff9d7158fc1800.statahead_batch_max=0 llite.lustre-ffff9d7158fc1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.850s user 0m0.352s sys 0m16.123s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d7158fc1800.statahead_batch_max=32 llite.lustre-ffff9d7158fc1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m39.083s user 0m0.328s sys 0m14.590s batched statahead RPC= hit count=9987 mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.batch_stats= snapshot_time: 1789461111.664575529 secs.nsecs start_time: 1789461001.121533701 secs.nsecs elapsed_time: 110.543041828 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.batch_stats=0 llite.lustre-ffff9d7158fc1800.statahead_max=2048 llite.lustre-ffff9d7158fc1800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.batch_stats=0 total: 100 open/close in 2.67 seconds: 37.50 ops/second llite.lustre-ffff9d7158fc1800.statahead_max=0 101 real 0m1.927s user 0m0.007s sys 0m0.344s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9d7158fc1800.statahead_max=2048 llite.lustre-ffff9d7158fc1800.statahead_max=2048 101 real 0m0.819s user 0m0.007s sys 0m0.179s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10086 miss_total: 14 swrong=0 ewrong=0 - open/close 709 (time 1789461441.61 total 10.01 last 70.86) total: 900 open/close in 13.03 seconds: 69.08 ops/second llite.lustre-ffff9d7158fc1800.statahead_max=0 1001 real 0m21.593s user 0m0.043s sys 0m3.073s 'ls -l' 1000 files without statahead: 21 sec llite.lustre-ffff9d7158fc1800.statahead_max=2048 llite.lustre-ffff9d7158fc1800.statahead_max=2048 1001 real 0m6.235s user 0m0.040s sys 0m1.601s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9d7158fc1800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11085 miss_total: 15 mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.batch_stats= snapshot_time: 1789461505.767727976 secs.nsecs start_time: 1789461115.160843365 secs.nsecs elapsed_time: 390.606884611 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 687 (time 1789461518.38 total 10.00 last 68.67) - open/close 1379 (time 1789461528.39 total 20.02 last 69.08) - open/close 2062 (time 1789461538.41 total 30.03 last 68.21) - open/close 2788 (time 1789461548.41 total 40.04 last 72.55) - open/close 3489 (time 1789461558.42 total 50.05 last 70.03) - open/close 4183 (time 1789461568.43 total 60.05 last 69.37) - open/close 4904 (time 1789461578.43 total 70.06 last 72.05) - open/close 5618 (time 1789461588.44 total 80.06 last 71.38) - open/close 6266 (time 1789461598.44 total 90.07 last 64.77) - open/close 6890 (time 1789461608.45 total 100.07 last 62.36) - open/close 7545 (time 1789461618.46 total 110.09 last 65.41) - open/close 8249 (time 1789461628.46 total 120.09 last 70.39) - open/close 8948 (time 1789461638.47 total 130.10 last 69.86) - open/close 9661 (time 1789461648.48 total 140.11 last 71.22) total: 10000 open/close in 145.35 seconds: 68.80 ops/second llite.lustre-ffff9d7158fc1800.statahead_batch_max=0 llite.lustre-ffff9d7158fc1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.134s user 0m0.350s sys 0m15.653s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9d7158fc1800.statahead_batch_max=256 llite.lustre-ffff9d7158fc1800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.260s user 0m0.299s sys 0m15.733s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9d7158fc1800.batch_stats= snapshot_time: 1789461786.381911605 secs.nsecs start_time: 1789461726.789710841 secs.nsecs elapsed_time: 59.592200764 secs.nsecs subreqs per batch batches %% cum %% 1: 103 47 47 2: 26 11 59 4: 10 4 64 8: 12 5 69 16: 10 4 74 32: 13 5 80 64: 7 3 83 128: 1 0 83 256: 35 16 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9d7158fc1800.statahead_batch_max=64 llite.lustre-ffff9d7158fc1800.statahead_max=128