== sanity test 123ad: Verify batching statahead works correctly ========================================================== 14:28:48 (1780338528) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.batch_stats=0 llite.lustre-ffff8a3447ef4800.statahead_max=32 llite.lustre-ffff8a3447ef4800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.batch_stats=0 total: 100 open/close in 3.19 seconds: 31.33 ops/second llite.lustre-ffff8a3447ef4800.statahead_max=0 101 real 0m2.894s user 0m0.006s sys 0m0.354s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8a3447ef4800.statahead_max=32 llite.lustre-ffff8a3447ef4800.statahead_max=32 101 real 0m1.073s user 0m0.004s sys 0m0.184s 'ls -l' 100 files with statahead: 1 sec statahead total: 142 statahead wrong: 2 agl total: 131 list_total: 29 fname_total: 86 hit_total: 4131 miss_total: 766 swrong=2 ewrong=2 - open/close 421 (time 1780338558.63 total 10.00 last 42.08) - open/close 881 (time 1780338568.65 total 20.03 last 45.90) total: 900 open/close in 20.34 seconds: 44.25 ops/second llite.lustre-ffff8a3447ef4800.statahead_max=0 1001 real 0m24.535s user 0m0.035s sys 0m3.269s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff8a3447ef4800.statahead_max=32 llite.lustre-ffff8a3447ef4800.statahead_max=32 1001 real 0m10.094s user 0m0.036s sys 0m1.611s 'ls -l' 1000 files with statahead: 10 sec statahead total: 143 statahead wrong: 2 agl total: 132 list_total: 30 fname_total: 86 hit_total: 5130 miss_total: 767 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff8a3447ef4800.statahead_stats= statahead total: 143 statahead wrong: 2 agl total: 132 list_total: 30 fname_total: 86 hit_total: 5130 miss_total: 767 mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.batch_stats= snapshot_time: 1780338638.456837603 secs.nsecs start_time: 1780338533.040693529 secs.nsecs elapsed_time: 105.416144074 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 673 (time 1780338650.95 total 10.02 last 67.20) - open/close 1251 (time 1780338660.95 total 20.02 last 57.79) - open/close 1831 (time 1780338670.96 total 30.02 last 57.96) - open/close 2466 (time 1780338680.96 total 40.02 last 63.50) - open/close 3176 (time 1780338690.97 total 50.04 last 70.90) - open/close 3897 (time 1780338700.99 total 60.06 last 71.96) - open/close 4468 (time 1780338711.00 total 70.07 last 57.02) - open/close 5103 (time 1780338721.02 total 80.08 last 63.42) - open/close 5700 (time 1780338731.02 total 90.09 last 59.65) - open/close 6362 (time 1780338741.03 total 100.09 last 66.18) - open/close 6981 (time 1780338751.03 total 110.09 last 61.89) - open/close 7587 (time 1780338761.04 total 120.10 last 60.55) - open/close 8289 (time 1780338771.05 total 130.12 last 70.11) - open/close 8931 (time 1780338781.09 total 140.16 last 63.95) - open/close 9583 (time 1780338791.09 total 150.16 last 65.17) total: 10000 open/close in 156.98 seconds: 63.70 ops/second llite.lustre-ffff8a3447ef4800.statahead_batch_max=0 llite.lustre-ffff8a3447ef4800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.574s user 0m0.325s sys 0m16.653s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8a3447ef4800.statahead_batch_max=32 llite.lustre-ffff8a3447ef4800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m44.447s user 0m0.328s sys 0m15.288s batched statahead RPC= hit count=9982 mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.batch_stats= snapshot_time: 1780338984.668026679 secs.nsecs start_time: 1780338867.228017006 secs.nsecs elapsed_time: 117.440009673 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.batch_stats=0 llite.lustre-ffff8a3447ef4800.statahead_max=2048 llite.lustre-ffff8a3447ef4800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.batch_stats=0 total: 100 open/close in 3.08 seconds: 32.44 ops/second llite.lustre-ffff8a3447ef4800.statahead_max=0 101 real 0m2.704s user 0m0.006s sys 0m0.323s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8a3447ef4800.statahead_max=2048 llite.lustre-ffff8a3447ef4800.statahead_max=2048 101 real 0m0.840s user 0m0.005s sys 0m0.185s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10081 miss_total: 19 swrong=0 ewrong=0 - open/close 582 (time 1780339344.75 total 10.02 last 58.10) total: 900 open/close in 15.37 seconds: 58.54 ops/second llite.lustre-ffff8a3447ef4800.statahead_max=0 1001 real 0m24.092s user 0m0.032s sys 0m2.962s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff8a3447ef4800.statahead_max=2048 llite.lustre-ffff8a3447ef4800.statahead_max=2048 1001 real 0m7.707s user 0m0.033s sys 0m1.436s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff8a3447ef4800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.batch_stats= snapshot_time: 1780339422.288420752 secs.nsecs start_time: 1780338987.140940769 secs.nsecs elapsed_time: 435.147479983 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 1 9 27 32: 3 27 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 658 (time 1780339435.11 total 10.01 last 65.74) - open/close 1392 (time 1780339445.12 total 20.02 last 73.35) - open/close 2067 (time 1780339455.13 total 30.02 last 67.45) - open/close 2653 (time 1780339465.13 total 40.02 last 58.59) - open/close 3376 (time 1780339475.14 total 50.04 last 72.21) - open/close 4022 (time 1780339485.15 total 60.05 last 64.53) - open/close 4656 (time 1780339495.16 total 70.06 last 63.34) - open/close 5285 (time 1780339505.16 total 80.06 last 62.90) - open/close 5841 (time 1780339515.17 total 90.06 last 55.56) - open/close 6368 (time 1780339525.18 total 100.07 last 52.65) - open/close 6959 (time 1780339535.19 total 110.08 last 59.03) - open/close 7570 (time 1780339545.20 total 120.09 last 61.04) - open/close 8267 (time 1780339555.22 total 130.11 last 69.59) - open/close 8901 (time 1780339565.22 total 140.12 last 63.36) - open/close 9531 (time 1780339575.27 total 150.16 last 62.72) total: 10000 open/close in 157.36 seconds: 63.55 ops/second llite.lustre-ffff8a3447ef4800.statahead_batch_max=0 llite.lustre-ffff8a3447ef4800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.732s user 0m0.327s sys 0m16.511s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8a3447ef4800.statahead_batch_max=256 llite.lustre-ffff8a3447ef4800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.679s user 0m0.288s sys 0m15.728s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8a3447ef4800.batch_stats= snapshot_time: 1780339730.266834866 secs.nsecs start_time: 1780339657.070272297 secs.nsecs elapsed_time: 73.196562569 secs.nsecs subreqs per batch batches % cum % 1: 259 53 53 2: 62 12 66 4: 66 13 80 8: 28 5 86 16: 15 3 89 32: 12 2 91 64: 6 1 92 128: 1 0 93 256: 33 6 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8a3447ef4800.statahead_batch_max=64 llite.lustre-ffff8a3447ef4800.statahead_max=128