== sanity test 123ad: Verify batching statahead works correctly ========================================================== 00:39:46 (1787114386) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.batch_stats=0 llite.lustre-ffff8aaa4a047800.statahead_max=32 llite.lustre-ffff8aaa4a047800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.31 seconds: 43.22 ops/second llite.lustre-ffff8aaa4a047800.statahead_max=0 101 real 0m1.908s user 0m0.006s sys 0m0.324s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8aaa4a047800.statahead_max=32 llite.lustre-ffff8aaa4a047800.statahead_max=32 101 real 0m0.964s user 0m0.004s sys 0m0.198s 'ls -l' 100 files with statahead: 1 sec statahead total: 121 statahead wrong: 2 agl total: 116 list_total: 30 fname_total: 91 hit_total: 4386 miss_total: 1273 swrong=2 ewrong=2 total: 900 open/close in 9.27 seconds: 97.06 ops/second llite.lustre-ffff8aaa4a047800.statahead_max=0 1001 real 0m22.627s user 0m0.035s sys 0m3.261s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff8aaa4a047800.statahead_max=32 llite.lustre-ffff8aaa4a047800.statahead_max=32 1001 real 0m9.269s user 0m0.036s sys 0m1.473s 'ls -l' 1000 files with statahead: 9 sec statahead total: 122 statahead wrong: 2 agl total: 117 list_total: 31 fname_total: 91 hit_total: 5384 miss_total: 1275 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff8aaa4a047800.statahead_stats= statahead total: 122 statahead wrong: 2 agl total: 117 list_total: 31 fname_total: 91 hit_total: 5384 miss_total: 1275 mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.batch_stats= snapshot_time: 1787114474.152841680 secs.nsecs start_time: 1787114391.018651775 secs.nsecs elapsed_time: 83.134189905 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 1 2 5 8: 4 10 15 16: 20 51 66 32: 13 33 100 mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.batch_stats= snapshot_time: 1787114474.153814155 secs.nsecs start_time: 1787114391.018738315 secs.nsecs elapsed_time: 83.135075840 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 3 7 10 8: 1 2 12 16: 20 51 64 32: 14 35 100 - open/close 980 (time 1787114486.69 total 10.00 last 97.96) - open/close 1849 (time 1787114496.69 total 20.01 last 86.89) - open/close 2864 (time 1787114506.69 total 30.01 last 101.48) - open/close 3664 (time 1787114516.69 total 40.01 last 79.98) - open/close 4520 (time 1787114526.70 total 50.01 last 85.58) - open/close 5362 (time 1787114536.70 total 60.02 last 84.14) - open/close 6249 (time 1787114546.71 total 70.03 last 88.64) - open/close 7140 (time 1787114556.71 total 80.03 last 89.09) - open/close 7972 (time 1787114566.72 total 90.04 last 83.10) - open/close 8778 (time 1787114576.74 total 100.05 last 80.48) - open/close 9613 (time 1787114586.75 total 110.06 last 83.43) total: 10000 open/close in 115.07 seconds: 86.90 ops/second llite.lustre-ffff8aaa4a047800.statahead_batch_max=0 llite.lustre-ffff8aaa4a047800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.stats=clear mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.613s user 0m0.299s sys 0m15.603s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8aaa4a047800.statahead_batch_max=32 llite.lustre-ffff8aaa4a047800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.stats=clear mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m49.018s user 0m0.280s sys 0m14.914s batched statahead RPC= hit count=9985 mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.batch_stats= snapshot_time: 1787114793.145360783 secs.nsecs start_time: 1787114664.656972672 secs.nsecs elapsed_time: 128.488388111 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.batch_stats= snapshot_time: 1787114793.145417104 secs.nsecs start_time: 1787114664.657179517 secs.nsecs elapsed_time: 128.488237587 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.batch_stats=0 llite.lustre-ffff8aaa4a047800.statahead_max=2048 llite.lustre-ffff8aaa4a047800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 2.64 seconds: 37.87 ops/second llite.lustre-ffff8aaa4a047800.statahead_max=0 101 real 0m2.682s user 0m0.004s sys 0m0.351s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8aaa4a047800.statahead_max=2048 llite.lustre-ffff8aaa4a047800.statahead_max=2048 101 real 0m1.102s user 0m0.004s sys 0m0.191s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10084 miss_total: 16 swrong=0 ewrong=0 - open/close 784 (time 1787115111.54 total 10.01 last 78.30) total: 900 open/close in 11.84 seconds: 75.99 ops/second llite.lustre-ffff8aaa4a047800.statahead_max=0 1001 real 0m21.566s user 0m0.035s sys 0m3.152s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff8aaa4a047800.statahead_max=2048 llite.lustre-ffff8aaa4a047800.statahead_max=2048 1001 real 0m8.213s user 0m0.030s sys 0m1.504s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff8aaa4a047800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11083 miss_total: 17 mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.batch_stats= snapshot_time: 1787115178.187288304 secs.nsecs start_time: 1787114794.903222123 secs.nsecs elapsed_time: 383.284066181 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 14 14 8: 3 42 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.batch_stats= snapshot_time: 1787115178.187415511 secs.nsecs start_time: 1787114794.906553242 secs.nsecs elapsed_time: 383.280862269 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 3 42 42 8: 1 14 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 - open/close 700 (time 1787115190.81 total 10.02 last 69.88) - open/close 1430 (time 1787115200.83 total 20.03 last 72.90) - open/close 2401 (time 1787115210.83 total 30.04 last 97.05) - open/close 3294 (time 1787115220.84 total 40.05 last 89.20) - open/close 3975 (time 1787115230.84 total 50.05 last 68.09) - open/close 4686 (time 1787115240.86 total 60.06 last 70.99) - open/close 5492 (time 1787115250.86 total 70.06 last 80.59) - open/close 6444 (time 1787115260.86 total 80.07 last 95.17) - open/close 7387 (time 1787115270.87 total 90.08 last 94.20) - open/close 8397 (time 1787115280.90 total 100.11 last 100.68) - open/close 9219 (time 1787115290.91 total 110.12 last 82.15) total: 10000 open/close in 119.92 seconds: 83.39 ops/second llite.lustre-ffff8aaa4a047800.statahead_batch_max=0 llite.lustre-ffff8aaa4a047800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.stats=clear mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.928s user 0m0.313s sys 0m15.991s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8aaa4a047800.statahead_batch_max=256 llite.lustre-ffff8aaa4a047800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.stats=clear mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.008s user 0m0.271s sys 0m16.146s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8aaa4a047800.batch_stats= snapshot_time: 1787115443.825794224 secs.nsecs start_time: 1787115376.736890102 secs.nsecs elapsed_time: 67.088904122 secs.nsecs subreqs per batch batches %% cum %% 1: 184 48 48 2: 50 13 61 4: 47 12 74 8: 24 6 80 16: 9 2 83 32: 22 5 88 64: 7 1 90 128: 2 0 91 256: 33 8 100 mdc.lustre-MDT0001-mdc-ffff8aaa4a047800.batch_stats= snapshot_time: 1787115443.825865506 secs.nsecs start_time: 1787115376.737048645 secs.nsecs elapsed_time: 67.088816861 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8aaa4a047800.statahead_batch_max=64 llite.lustre-ffff8aaa4a047800.statahead_max=128