== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:12:21 (1790946741) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.batch_stats=0 llite.lustre-ffff9ac802bc7800.statahead_max=32 llite.lustre-ffff9ac802bc7800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.batch_stats=0 total: 100 open/close in 3.61 seconds: 27.70 ops/second llite.lustre-ffff9ac802bc7800.statahead_max=0 101 real 0m2.743s user 0m0.007s sys 0m0.338s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9ac802bc7800.statahead_max=32 llite.lustre-ffff9ac802bc7800.statahead_max=32 101 real 0m1.288s user 0m0.005s sys 0m0.145s 'ls -l' 100 files with statahead: 1 sec statahead total: 127 statahead wrong: 2 agl total: 121 list_total: 29 fname_total: 65 hit_total: 3869 miss_total: 1654 swrong=2 ewrong=2 - open/close 617 (time 1790946770.64 total 10.00 last 61.69) total: 900 open/close in 14.43 seconds: 62.35 ops/second llite.lustre-ffff9ac802bc7800.statahead_max=0 1001 real 0m23.292s user 0m0.044s sys 0m2.867s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff9ac802bc7800.statahead_max=32 llite.lustre-ffff9ac802bc7800.statahead_max=32 1001 real 0m11.212s user 0m0.024s sys 0m1.409s 'ls -l' 1000 files with statahead: 11 sec statahead total: 128 statahead wrong: 2 agl total: 122 list_total: 30 fname_total: 65 hit_total: 4867 miss_total: 1656 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff9ac802bc7800.statahead_stats= statahead total: 128 statahead wrong: 2 agl total: 122 list_total: 30 fname_total: 65 hit_total: 4867 miss_total: 1656 mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.batch_stats= snapshot_time: 1790946849.590860407 secs.nsecs start_time: 1790946746.490019741 secs.nsecs elapsed_time: 103.100840666 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 1 2 12 32: 34 87 100 - open/close 540 (time 1790946862.82 total 10.03 last 53.86) - open/close 959 (time 1790946872.82 total 20.03 last 41.87) - open/close 1507 (time 1790946882.83 total 30.04 last 54.75) - open/close 2023 (time 1790946892.83 total 40.04 last 51.59) - open/close 2600 (time 1790946902.84 total 50.05 last 57.64) - open/close 3086 (time 1790946912.86 total 60.07 last 48.53) - open/close 3705 (time 1790946922.86 total 70.07 last 61.90) - open/close 4297 (time 1790946932.86 total 80.07 last 59.17) - open/close 4868 (time 1790946942.87 total 90.08 last 57.05) - open/close 5429 (time 1790946952.88 total 100.09 last 56.08) - open/close 6043 (time 1790946962.90 total 110.11 last 61.25) - open/close 6657 (time 1790946972.91 total 120.12 last 61.37) - open/close 7306 (time 1790946982.92 total 130.13 last 64.84) - open/close 7885 (time 1790946992.92 total 140.13 last 57.88) - open/close 8479 (time 1790947002.92 total 150.14 last 59.36) - open/close 9047 (time 1790947012.94 total 160.15 last 56.74) - open/close 9595 (time 1790947022.95 total 170.16 last 54.74) total: 10000 open/close in 176.84 seconds: 56.55 ops/second llite.lustre-ffff9ac802bc7800.statahead_batch_max=0 llite.lustre-ffff9ac802bc7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.284s user 0m0.333s sys 0m14.423s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ac802bc7800.statahead_batch_max=32 llite.lustre-ffff9ac802bc7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.stats=clear ls -l files in d123ad.sanity: 10001 real 2m3.377s user 0m0.344s sys 0m13.671s batched statahead RPC= hit count=9972 mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.batch_stats= snapshot_time: 1790947240.352952963 secs.nsecs start_time: 1790947105.003906900 secs.nsecs elapsed_time: 135.349046063 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.batch_stats=0 llite.lustre-ffff9ac802bc7800.statahead_max=2048 llite.lustre-ffff9ac802bc7800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.batch_stats=0 total: 100 open/close in 4.15 seconds: 24.10 ops/second llite.lustre-ffff9ac802bc7800.statahead_max=0 101 real 0m2.739s user 0m0.007s sys 0m0.329s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9ac802bc7800.statahead_max=2048 llite.lustre-ffff9ac802bc7800.statahead_max=2048 101 real 0m1.078s user 0m0.007s sys 0m0.182s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10071 miss_total: 29 swrong=0 ewrong=0 - open/close 641 (time 1790947633.44 total 10.01 last 64.03) total: 900 open/close in 14.33 seconds: 62.81 ops/second llite.lustre-ffff9ac802bc7800.statahead_max=0 1001 real 0m25.764s user 0m0.041s sys 0m3.114s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff9ac802bc7800.statahead_max=2048 llite.lustre-ffff9ac802bc7800.statahead_max=2048 1001 real 0m7.089s user 0m0.026s sys 0m1.319s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11070 miss_total: 30 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 28 seconds llite.lustre-ffff9ac802bc7800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11070 miss_total: 30 mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.batch_stats= snapshot_time: 1790947713.259343578 secs.nsecs start_time: 1790947243.470899214 secs.nsecs elapsed_time: 469.788444364 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 22 22 16: 2 22 44 32: 0 0 44 64: 0 0 44 128: 1 11 55 256: 4 44 100 - open/close 568 (time 1790947726.41 total 10.01 last 56.77) - open/close 1140 (time 1790947736.41 total 20.01 last 57.18) - open/close 1637 (time 1790947746.43 total 30.03 last 49.61) - open/close 2095 (time 1790947756.45 total 40.05 last 45.70) - open/close 2517 (time 1790947766.47 total 50.06 last 42.15) - open/close 3022 (time 1790947776.48 total 60.07 last 50.44) - open/close 3458 (time 1790947786.50 total 70.10 last 43.50) - open/close 3948 (time 1790947796.50 total 80.10 last 49.00) - open/close 4501 (time 1790947806.51 total 90.11 last 55.24) - open/close 4969 (time 1790947816.52 total 100.11 last 46.78) - open/close 5351 (time 1790947826.57 total 110.17 last 38.00) - open/close 5751 (time 1790947836.60 total 120.20 last 39.88) - open/close 6219 (time 1790947846.60 total 130.20 last 46.79) - open/close 6744 (time 1790947856.61 total 140.21 last 52.44) - open/close 7331 (time 1790947866.62 total 150.21 last 58.66) - open/close 7865 (time 1790947876.63 total 160.22 last 53.37) - open/close 8440 (time 1790947886.63 total 170.23 last 57.47) - open/close 9039 (time 1790947896.64 total 180.23 last 59.86) - open/close 9600 (time 1790947906.65 total 190.24 last 56.05) total: 10000 open/close in 197.62 seconds: 50.60 ops/second llite.lustre-ffff9ac802bc7800.statahead_batch_max=0 llite.lustre-ffff9ac802bc7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.641s user 0m0.352s sys 0m14.697s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ac802bc7800.statahead_batch_max=256 llite.lustre-ffff9ac802bc7800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.117s user 0m0.297s sys 0m13.841s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9ac802bc7800.batch_stats= snapshot_time: 1790948070.987328684 secs.nsecs start_time: 1790947998.871597462 secs.nsecs elapsed_time: 72.115731222 secs.nsecs subreqs per batch batches %% cum %% 1: 134 47 47 2: 41 14 62 4: 24 8 70 8: 13 4 75 16: 15 5 80 32: 18 6 87 64: 0 0 87 128: 1 0 87 256: 35 12 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9ac802bc7800.statahead_batch_max=64 llite.lustre-ffff9ac802bc7800.statahead_max=128