== sanity test 123ad: Verify batching statahead works correctly ========================================================== 00:09:17 (1787371757) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff97f742ebd000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff97f742ebd000.batch_stats=0 llite.lustre-ffff97f742ebd000.statahead_max=32 llite.lustre-ffff97f742ebd000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff97f742ebd000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff97f742ebd000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.19 seconds: 45.60 ops/second llite.lustre-ffff97f742ebd000.statahead_max=0 101 real 0m2.022s user 0m0.005s sys 0m0.289s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff97f742ebd000.statahead_max=32 llite.lustre-ffff97f742ebd000.statahead_max=32 101 real 0m0.974s user 0m0.002s sys 0m0.183s 'ls -l' 100 files with statahead: 1 sec statahead total: 126 statahead wrong: 1 agl total: 120 list_total: 31 fname_total: 92 hit_total: 3834 miss_total: 266 swrong=1 ewrong=1 - open/close 868 (time 1787371782.96 total 10.00 last 86.80) total: 900 open/close in 10.26 seconds: 87.73 ops/second llite.lustre-ffff97f742ebd000.statahead_max=0 1001 real 0m18.176s user 0m0.028s sys 0m2.757s 'ls -l' 1000 files without statahead: 18 sec llite.lustre-ffff97f742ebd000.statahead_max=32 llite.lustre-ffff97f742ebd000.statahead_max=32 1001 real 0m7.855s user 0m0.027s sys 0m1.378s 'ls -l' 1000 files with statahead: 8 sec statahead total: 127 statahead wrong: 1 agl total: 121 list_total: 32 fname_total: 92 hit_total: 4832 miss_total: 268 swrong=1 ewrong=1 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 17 seconds llite.lustre-ffff97f742ebd000.statahead_stats= statahead total: 127 statahead wrong: 1 agl total: 121 list_total: 32 fname_total: 92 hit_total: 4832 miss_total: 268 mdc.lustre-MDT0000-mdc-ffff97f742ebd000.batch_stats= snapshot_time: 1787371837.999697617 secs.nsecs start_time: 1787371761.918238327 secs.nsecs elapsed_time: 76.081459290 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 3 7 10 8: 2 5 15 16: 21 53 69 32: 12 30 100 mdc.lustre-MDT0001-mdc-ffff97f742ebd000.batch_stats= snapshot_time: 1787371837.999756606 secs.nsecs start_time: 1787371761.918316697 secs.nsecs elapsed_time: 76.081439909 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 5 5 8: 5 12 17 16: 19 48 66 32: 13 33 100 - open/close 904 (time 1787371850.47 total 10.00 last 90.36) - open/close 1910 (time 1787371860.47 total 20.01 last 100.59) - open/close 2784 (time 1787371870.47 total 30.01 last 87.40) - open/close 3719 (time 1787371880.48 total 40.01 last 93.45) - open/close 4693 (time 1787371890.48 total 50.01 last 97.39) - open/close 5608 (time 1787371900.48 total 60.02 last 91.45) - open/close 6544 (time 1787371910.49 total 70.03 last 93.48) - open/close 7478 (time 1787371920.50 total 80.04 last 93.31) - open/close 8294 (time 1787371930.51 total 90.04 last 81.59) - open/close 9232 (time 1787371940.51 total 100.05 last 93.77) total: 10000 open/close in 107.99 seconds: 92.60 ops/second llite.lustre-ffff97f742ebd000.statahead_batch_max=0 llite.lustre-ffff97f742ebd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97f742ebd000.stats=clear mdc.lustre-MDT0001-mdc-ffff97f742ebd000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.679s user 0m0.300s sys 0m15.389s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff97f742ebd000.statahead_batch_max=32 llite.lustre-ffff97f742ebd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97f742ebd000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff97f742ebd000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff97f742ebd000.stats=clear mdc.lustre-MDT0001-mdc-ffff97f742ebd000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m27.795s user 0m0.282s sys 0m13.901s batched statahead RPC= hit count=9986 mdc.lustre-MDT0000-mdc-ffff97f742ebd000.batch_stats= snapshot_time: 1787372110.476412335 secs.nsecs start_time: 1787372012.187985729 secs.nsecs elapsed_time: 98.288426606 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff97f742ebd000.batch_stats= snapshot_time: 1787372110.476483057 secs.nsecs start_time: 1787372012.188169277 secs.nsecs elapsed_time: 98.288313780 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff97f742ebd000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff97f742ebd000.batch_stats=0 llite.lustre-ffff97f742ebd000.statahead_max=2048 llite.lustre-ffff97f742ebd000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff97f742ebd000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff97f742ebd000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.02 seconds: 49.49 ops/second llite.lustre-ffff97f742ebd000.statahead_max=0 101 real 0m1.534s user 0m0.005s sys 0m0.307s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff97f742ebd000.statahead_max=2048 llite.lustre-ffff97f742ebd000.statahead_max=2048 101 real 0m0.730s user 0m0.006s sys 0m0.177s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10085 miss_total: 15 swrong=0 ewrong=0 total: 900 open/close in 9.56 seconds: 94.15 ops/second llite.lustre-ffff97f742ebd000.statahead_max=0 1001 real 0m18.445s user 0m0.031s sys 0m2.814s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff97f742ebd000.statahead_max=2048 llite.lustre-ffff97f742ebd000.statahead_max=2048 1001 real 0m6.048s user 0m0.030s sys 0m1.426s 'ls -l' 1000 files with statahead: 6 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff97f742ebd000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11084 miss_total: 16 mdc.lustre-MDT0000-mdc-ffff97f742ebd000.batch_stats= snapshot_time: 1787372435.800076526 secs.nsecs start_time: 1787372112.649052785 secs.nsecs elapsed_time: 323.151023741 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 3 33 33 8: 2 22 55 16: 1 11 66 32: 0 0 66 64: 1 11 77 128: 0 0 77 256: 2 22 100 mdc.lustre-MDT0001-mdc-ffff97f742ebd000.batch_stats= snapshot_time: 1787372435.800136623 secs.nsecs start_time: 1787372112.649186100 secs.nsecs elapsed_time: 323.150950523 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 11 11 8: 4 44 55 16: 1 11 66 32: 1 11 77 64: 0 0 77 128: 0 0 77 256: 2 22 100 - open/close 809 (time 1787372448.33 total 10.01 last 80.86) - open/close 1709 (time 1787372458.33 total 20.01 last 89.94) - open/close 2596 (time 1787372468.34 total 30.02 last 88.65) - open/close 3580 (time 1787372478.34 total 40.02 last 98.38) - open/close 4501 (time 1787372488.35 total 50.02 last 92.06) - open/close 5446 (time 1787372498.35 total 60.03 last 94.43) - open/close 6270 (time 1787372508.36 total 70.04 last 82.34) - open/close 7225 (time 1787372518.37 total 80.05 last 95.44) - open/close 8201 (time 1787372528.37 total 90.05 last 97.53) - open/close 9173 (time 1787372538.38 total 100.06 last 97.14) total: 10000 open/close in 109.44 seconds: 91.37 ops/second llite.lustre-ffff97f742ebd000.statahead_batch_max=0 llite.lustre-ffff97f742ebd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97f742ebd000.stats=clear mdc.lustre-MDT0001-mdc-ffff97f742ebd000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.279s user 0m0.322s sys 0m15.732s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff97f742ebd000.statahead_batch_max=256 llite.lustre-ffff97f742ebd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff97f742ebd000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff97f742ebd000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff97f742ebd000.stats=clear mdc.lustre-MDT0001-mdc-ffff97f742ebd000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m53.183s user 0m0.336s sys 0m14.849s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff97f742ebd000.batch_stats= snapshot_time: 1787372681.575969618 secs.nsecs start_time: 1787372612.728790506 secs.nsecs elapsed_time: 68.847179112 secs.nsecs subreqs per batch batches %% cum %% 1: 97 41 41 2: 23 9 51 4: 22 9 60 8: 21 9 69 16: 9 3 73 32: 19 8 81 64: 5 2 84 128: 3 1 85 256: 34 14 100 mdc.lustre-MDT0001-mdc-ffff97f742ebd000.batch_stats= snapshot_time: 1787372681.576037433 secs.nsecs start_time: 1787372612.728867204 secs.nsecs elapsed_time: 68.847170229 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff97f742ebd000.statahead_batch_max=64 llite.lustre-ffff97f742ebd000.statahead_max=128