== sanity test 123ad: Verify batching statahead works correctly ========================================================== 04:56:56 (1784192216) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.batch_stats=0 llite.lustre-ffff8ffc46e8c000.statahead_max=32 llite.lustre-ffff8ffc46e8c000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.batch_stats=0 total: 100 open/close in 2.88 seconds: 34.68 ops/second llite.lustre-ffff8ffc46e8c000.statahead_max=0 101 real 0m3.162s user 0m0.006s sys 0m0.384s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8ffc46e8c000.statahead_max=32 llite.lustre-ffff8ffc46e8c000.statahead_max=32 101 real 0m1.558s user 0m0.006s sys 0m0.190s 'ls -l' 100 files with statahead: 2 sec statahead total: 126 statahead wrong: 2 agl total: 120 list_total: 30 fname_total: 80 hit_total: 3556 miss_total: 1718 swrong=2 ewrong=2 - open/close 570 (time 1784192250.38 total 10.02 last 56.90) total: 900 open/close in 15.46 seconds: 58.20 ops/second llite.lustre-ffff8ffc46e8c000.statahead_max=0 1001 real 0m26.700s user 0m0.038s sys 0m3.538s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff8ffc46e8c000.statahead_max=32 llite.lustre-ffff8ffc46e8c000.statahead_max=32 1001 real 0m13.499s user 0m0.035s sys 0m1.695s 'ls -l' 1000 files with statahead: 14 sec statahead total: 127 statahead wrong: 2 agl total: 121 list_total: 31 fname_total: 80 hit_total: 4550 miss_total: 1724 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff8ffc46e8c000.statahead_stats= statahead total: 127 statahead wrong: 2 agl total: 121 list_total: 31 fname_total: 80 hit_total: 4550 miss_total: 1724 mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.batch_stats= snapshot_time: 1784192337.579221469 secs.nsecs start_time: 1784192224.501805593 secs.nsecs elapsed_time: 113.077415876 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 5 5 16: 3 7 12 32: 34 87 100 - open/close 628 (time 1784192351.18 total 10.01 last 62.75) - open/close 1176 (time 1784192361.19 total 20.02 last 54.76) - open/close 1857 (time 1784192371.20 total 30.03 last 67.99) - open/close 2485 (time 1784192381.21 total 40.04 last 62.77) - open/close 3061 (time 1784192391.22 total 50.05 last 57.53) - open/close 3566 (time 1784192401.23 total 60.06 last 50.48) - open/close 4127 (time 1784192411.23 total 70.06 last 56.06) - open/close 4772 (time 1784192421.26 total 80.09 last 64.35) - open/close 5398 (time 1784192431.26 total 90.09 last 62.58) - open/close 6003 (time 1784192441.27 total 100.10 last 60.43) - open/close 6515 (time 1784192451.28 total 110.11 last 51.17) - open/close 7102 (time 1784192461.28 total 120.11 last 58.67) - open/close 7606 (time 1784192471.29 total 130.12 last 50.37) - open/close 8255 (time 1784192481.36 total 140.19 last 64.43) - open/close 8787 (time 1784192491.37 total 150.20 last 53.13) - open/close 9298 (time 1784192501.39 total 160.21 last 51.04) - open/close 9816 (time 1784192511.40 total 170.23 last 51.74) total: 10000 open/close in 174.44 seconds: 57.33 ops/second llite.lustre-ffff8ffc46e8c000.statahead_batch_max=0 llite.lustre-ffff8ffc46e8c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.212s user 0m0.403s sys 0m18.244s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8ffc46e8c000.statahead_batch_max=32 llite.lustre-ffff8ffc46e8c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.stats=clear ls -l files in d123ad.sanity: 10001 real 2m7.462s user 0m0.360s sys 0m16.686s batched statahead RPC= hit count=9974 mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.batch_stats= snapshot_time: 1784192746.441376759 secs.nsecs start_time: 1784192602.115888297 secs.nsecs elapsed_time: 144.325488462 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 0 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.batch_stats=0 llite.lustre-ffff8ffc46e8c000.statahead_max=2048 llite.lustre-ffff8ffc46e8c000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.batch_stats=0 total: 100 open/close in 3.23 seconds: 30.94 ops/second llite.lustre-ffff8ffc46e8c000.statahead_max=0 101 real 0m2.288s user 0m0.009s sys 0m0.379s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8ffc46e8c000.statahead_max=2048 llite.lustre-ffff8ffc46e8c000.statahead_max=2048 101 real 0m0.862s user 0m0.004s sys 0m0.186s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10073 miss_total: 27 swrong=0 ewrong=0 - open/close 548 (time 1784193112.09 total 10.01 last 54.76) total: 900 open/close in 15.69 seconds: 57.38 ops/second llite.lustre-ffff8ffc46e8c000.statahead_max=0 1001 real 0m27.290s user 0m0.041s sys 0m3.514s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff8ffc46e8c000.statahead_max=2048 llite.lustre-ffff8ffc46e8c000.statahead_max=2048 1001 real 0m10.717s user 0m0.036s sys 0m1.719s 'ls -l' 1000 files with statahead: 11 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 32 seconds llite.lustre-ffff8ffc46e8c000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11072 miss_total: 28 mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.batch_stats= snapshot_time: 1784193207.346445187 secs.nsecs start_time: 1784192749.404431576 secs.nsecs elapsed_time: 457.942013611 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 16 16 16: 2 16 33 32: 2 16 50 64: 1 8 58 128: 1 8 66 256: 4 33 100 - open/close 579 (time 1784193220.08 total 10.03 last 57.71) - open/close 1142 (time 1784193230.09 total 20.04 last 56.24) - open/close 1746 (time 1784193240.11 total 30.06 last 60.33) - open/close 2296 (time 1784193250.11 total 40.06 last 55.00) - open/close 2866 (time 1784193260.12 total 50.07 last 56.93) - open/close 3521 (time 1784193270.12 total 60.07 last 65.46) - open/close 4297 (time 1784193280.13 total 70.08 last 77.54) - open/close 5477 (time 1784193290.13 total 80.08 last 117.99) - open/close 6391 (time 1784193300.14 total 90.09 last 91.34) - open/close 6819 (time 1784193310.14 total 100.09 last 42.79) - open/close 7328 (time 1784193320.15 total 110.10 last 50.83) - open/close 7887 (time 1784193330.16 total 120.11 last 55.90) - open/close 8500 (time 1784193340.16 total 130.11 last 61.28) - open/close 9024 (time 1784193350.16 total 140.11 last 52.40) - open/close 9511 (time 1784193360.17 total 150.12 last 48.66) total: 10000 open/close in 159.23 seconds: 62.80 ops/second llite.lustre-ffff8ffc46e8c000.statahead_batch_max=0 llite.lustre-ffff8ffc46e8c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m11.862s user 0m0.444s sys 0m20.247s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8ffc46e8c000.statahead_batch_max=256 llite.lustre-ffff8ffc46e8c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.312s user 0m0.362s sys 0m18.074s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8ffc46e8c000.batch_stats= snapshot_time: 1784193533.305372868 secs.nsecs start_time: 1784193452.854567998 secs.nsecs elapsed_time: 80.450804870 secs.nsecs subreqs per batch batches %% cum %% 1: 231 52 52 2: 61 13 66 4: 50 11 77 8: 24 5 83 16: 19 4 87 32: 16 3 91 64: 5 1 92 128: 1 0 92 256: 33 7 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8ffc46e8c000.statahead_batch_max=64 llite.lustre-ffff8ffc46e8c000.statahead_max=128