== sanity test 123ad: Verify batching statahead works correctly ========================================================== 01:12:52 (1788844372) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff953bd30d4000.batch_stats=0 llite.lustre-ffff953bd30d4000.statahead_max=32 llite.lustre-ffff953bd30d4000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff953bd30d4000.batch_stats=0 total: 100 open/close in 2.61 seconds: 38.37 ops/second llite.lustre-ffff953bd30d4000.statahead_max=0 101 real 0m2.226s user 0m0.006s sys 0m0.305s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff953bd30d4000.statahead_max=32 llite.lustre-ffff953bd30d4000.statahead_max=32 101 real 0m0.826s user 0m0.004s sys 0m0.132s 'ls -l' 100 files with statahead: 0 sec statahead total: 122 statahead wrong: 2 agl total: 116 list_total: 30 fname_total: 75 hit_total: 4010 miss_total: 1434 swrong=2 ewrong=2 - open/close 677 (time 1788844400.42 total 10.02 last 67.59) total: 900 open/close in 13.03 seconds: 69.09 ops/second llite.lustre-ffff953bd30d4000.statahead_max=0 1001 real 0m22.433s user 0m0.027s sys 0m2.875s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff953bd30d4000.statahead_max=32 llite.lustre-ffff953bd30d4000.statahead_max=32 1001 real 0m10.800s user 0m0.029s sys 0m1.337s 'ls -l' 1000 files with statahead: 11 sec statahead total: 123 statahead wrong: 2 agl total: 117 list_total: 31 fname_total: 75 hit_total: 5008 miss_total: 1436 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff953bd30d4000.statahead_stats= statahead total: 123 statahead wrong: 2 agl total: 117 list_total: 31 fname_total: 75 hit_total: 5008 miss_total: 1436 mdc.lustre-MDT0000-mdc-ffff953bd30d4000.batch_stats= snapshot_time: 1788844479.836613642 secs.nsecs start_time: 1788844377.580249621 secs.nsecs elapsed_time: 102.256364021 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 1 2 5 8: 2 5 10 16: 2 5 15 32: 34 85 100 - open/close 650 (time 1788844493.23 total 10.02 last 64.87) - open/close 1228 (time 1788844503.23 total 20.02 last 57.79) - open/close 1915 (time 1788844513.24 total 30.02 last 68.68) - open/close 2611 (time 1788844523.25 total 40.04 last 69.51) - open/close 3381 (time 1788844533.26 total 50.05 last 76.92) - open/close 4141 (time 1788844543.27 total 60.06 last 75.91) - open/close 4871 (time 1788844553.28 total 70.07 last 72.93) - open/close 5678 (time 1788844563.29 total 80.08 last 80.64) - open/close 6390 (time 1788844573.29 total 90.08 last 71.19) - open/close 7110 (time 1788844583.31 total 100.10 last 71.87) - open/close 7862 (time 1788844593.31 total 110.10 last 75.18) - open/close 8598 (time 1788844603.32 total 120.11 last 73.55) - open/close 9405 (time 1788844613.32 total 130.11 last 80.65) total: 10000 open/close in 137.37 seconds: 72.80 ops/second llite.lustre-ffff953bd30d4000.statahead_batch_max=0 llite.lustre-ffff953bd30d4000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff953bd30d4000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m8.360s user 0m0.354s sys 0m14.001s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff953bd30d4000.statahead_batch_max=32 llite.lustre-ffff953bd30d4000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff953bd30d4000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff953bd30d4000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m54.995s user 0m0.310s sys 0m13.208s batched statahead RPC= hit count=9980 mdc.lustre-MDT0000-mdc-ffff953bd30d4000.batch_stats= snapshot_time: 1788844823.398784869 secs.nsecs start_time: 1788844697.767333814 secs.nsecs elapsed_time: 125.631451055 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff953bd30d4000.batch_stats=0 llite.lustre-ffff953bd30d4000.statahead_max=2048 llite.lustre-ffff953bd30d4000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff953bd30d4000.batch_stats=0 total: 100 open/close in 2.00 seconds: 50.12 ops/second llite.lustre-ffff953bd30d4000.statahead_max=0 101 real 0m1.790s user 0m0.006s sys 0m0.309s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff953bd30d4000.statahead_max=2048 llite.lustre-ffff953bd30d4000.statahead_max=2048 101 real 0m1.391s user 0m0.007s sys 0m0.188s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10079 miss_total: 21 swrong=0 ewrong=0 - open/close 723 (time 1788845141.94 total 10.02 last 72.18) total: 900 open/close in 13.06 seconds: 68.94 ops/second llite.lustre-ffff953bd30d4000.statahead_max=0 1001 real 0m20.670s user 0m0.028s sys 0m2.900s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff953bd30d4000.statahead_max=2048 llite.lustre-ffff953bd30d4000.statahead_max=2048 1001 real 0m7.327s user 0m0.034s sys 0m1.243s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff953bd30d4000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11078 miss_total: 22 mdc.lustre-MDT0000-mdc-ffff953bd30d4000.batch_stats= snapshot_time: 1788845209.920253137 secs.nsecs start_time: 1788844827.613173797 secs.nsecs elapsed_time: 382.307079340 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 25 25 16: 1 12 37 32: 0 0 37 64: 0 0 37 128: 1 12 50 256: 4 50 100 - open/close 722 (time 1788845222.65 total 10.01 last 72.14) - open/close 1480 (time 1788845232.66 total 20.02 last 75.71) - open/close 2204 (time 1788845242.66 total 30.02 last 72.38) - open/close 2994 (time 1788845252.67 total 40.03 last 78.96) - open/close 3659 (time 1788845262.69 total 50.05 last 66.33) - open/close 4453 (time 1788845272.69 total 60.05 last 79.38) - open/close 5186 (time 1788845282.71 total 70.07 last 73.15) - open/close 5961 (time 1788845292.73 total 80.08 last 77.42) - open/close 6729 (time 1788845302.74 total 90.10 last 76.69) - open/close 7514 (time 1788845312.74 total 100.10 last 78.50) - open/close 8218 (time 1788845322.75 total 110.11 last 70.34) - open/close 8914 (time 1788845332.75 total 120.11 last 69.58) - open/close 9633 (time 1788845342.75 total 130.11 last 71.90) total: 10000 open/close in 135.83 seconds: 73.62 ops/second llite.lustre-ffff953bd30d4000.statahead_batch_max=0 llite.lustre-ffff953bd30d4000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff953bd30d4000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.211s user 0m0.331s sys 0m14.777s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff953bd30d4000.statahead_batch_max=256 llite.lustre-ffff953bd30d4000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff953bd30d4000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff953bd30d4000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.689s user 0m0.318s sys 0m14.049s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff953bd30d4000.batch_stats= snapshot_time: 1788845476.685807161 secs.nsecs start_time: 1788845415.067162874 secs.nsecs elapsed_time: 61.618644287 secs.nsecs subreqs per batch batches %% cum %% 1: 151 53 53 2: 30 10 64 4: 19 6 70 8: 15 5 76 16: 14 4 81 32: 14 4 86 64: 4 1 87 128: 0 0 87 256: 35 12 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff953bd30d4000.statahead_batch_max=64 llite.lustre-ffff953bd30d4000.statahead_max=128