== sanity test 123ad: Verify batching statahead works correctly ========================================================== 20:08:12 (1787702892) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff98f2b518f800.batch_stats=0 llite.lustre-ffff98f2b518f800.statahead_max=32 llite.lustre-ffff98f2b518f800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff98f2b518f800.batch_stats=0 total: 100 open/close in 3.11 seconds: 32.14 ops/second llite.lustre-ffff98f2b518f800.statahead_max=0 101 real 0m2.353s user 0m0.007s sys 0m0.291s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff98f2b518f800.statahead_max=32 llite.lustre-ffff98f2b518f800.statahead_max=32 101 real 0m0.916s user 0m0.005s sys 0m0.152s 'ls -l' 100 files with statahead: 1 sec statahead total: 129 statahead wrong: 2 agl total: 123 list_total: 30 fname_total: 86 hit_total: 4035 miss_total: 1649 swrong=2 ewrong=2 - open/close 655 (time 1787702922.14 total 10.02 last 65.34) total: 900 open/close in 14.22 seconds: 63.29 ops/second llite.lustre-ffff98f2b518f800.statahead_max=0 1001 real 0m25.838s user 0m0.030s sys 0m3.002s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff98f2b518f800.statahead_max=32 llite.lustre-ffff98f2b518f800.statahead_max=32 1001 real 0m10.135s user 0m0.029s sys 0m1.337s 'ls -l' 1000 files with statahead: 10 sec statahead total: 130 statahead wrong: 2 agl total: 124 list_total: 31 fname_total: 86 hit_total: 5034 miss_total: 1650 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff98f2b518f800.statahead_stats= statahead total: 130 statahead wrong: 2 agl total: 124 list_total: 31 fname_total: 86 hit_total: 5034 miss_total: 1650 mdc.lustre-MDT0000-mdc-ffff98f2b518f800.batch_stats= snapshot_time: 1787703003.700295621 secs.nsecs start_time: 1787702897.949306427 secs.nsecs elapsed_time: 105.750989194 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 570 (time 1787703017.44 total 10.01 last 56.96) - open/close 1290 (time 1787703027.46 total 20.03 last 71.85) - open/close 1943 (time 1787703037.48 total 30.05 last 65.17) - open/close 2556 (time 1787703047.48 total 40.05 last 61.28) - open/close 3235 (time 1787703057.50 total 50.06 last 67.83) - open/close 3953 (time 1787703067.51 total 60.08 last 71.70) - open/close 4664 (time 1787703077.51 total 70.08 last 71.08) - open/close 5356 (time 1787703087.52 total 80.09 last 69.14) - open/close 6050 (time 1787703097.53 total 90.09 last 69.36) - open/close 6847 (time 1787703107.53 total 100.09 last 79.69) - open/close 7533 (time 1787703117.55 total 110.12 last 68.45) - open/close 8223 (time 1787703127.56 total 120.13 last 68.92) - open/close 8948 (time 1787703137.57 total 130.14 last 72.40) - open/close 9622 (time 1787703147.58 total 140.15 last 67.35) total: 10000 open/close in 145.43 seconds: 68.76 ops/second llite.lustre-ffff98f2b518f800.statahead_batch_max=0 llite.lustre-ffff98f2b518f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98f2b518f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m55.053s user 0m0.309s sys 0m14.723s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff98f2b518f800.statahead_batch_max=32 llite.lustre-ffff98f2b518f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98f2b518f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff98f2b518f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m39.318s user 0m0.339s sys 0m13.843s batched statahead RPC= hit count=9982 mdc.lustre-MDT0000-mdc-ffff98f2b518f800.batch_stats= snapshot_time: 1787703326.516100138 secs.nsecs start_time: 1787703216.171630418 secs.nsecs elapsed_time: 110.344469720 secs.nsecs subreqs per batch batches %% cum %% 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff98f2b518f800.batch_stats=0 llite.lustre-ffff98f2b518f800.statahead_max=2048 llite.lustre-ffff98f2b518f800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff98f2b518f800.batch_stats=0 total: 100 open/close in 3.14 seconds: 31.80 ops/second llite.lustre-ffff98f2b518f800.statahead_max=0 101 real 0m2.505s user 0m0.005s sys 0m0.314s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff98f2b518f800.statahead_max=2048 llite.lustre-ffff98f2b518f800.statahead_max=2048 101 real 0m0.961s user 0m0.005s sys 0m0.154s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10081 miss_total: 19 swrong=0 ewrong=0 - open/close 642 (time 1787703648.48 total 10.03 last 64.02) total: 900 open/close in 13.87 seconds: 64.88 ops/second llite.lustre-ffff98f2b518f800.statahead_max=0 1001 real 0m21.597s user 0m0.046s sys 0m2.856s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff98f2b518f800.statahead_max=2048 llite.lustre-ffff98f2b518f800.statahead_max=2048 1001 real 0m6.409s user 0m0.025s sys 0m1.408s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff98f2b518f800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 mdc.lustre-MDT0000-mdc-ffff98f2b518f800.batch_stats= snapshot_time: 1787703713.768820598 secs.nsecs start_time: 1787703329.309761659 secs.nsecs elapsed_time: 384.459058939 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 740 (time 1787703725.98 total 10.01 last 73.92) - open/close 1331 (time 1787703735.99 total 20.02 last 59.04) - open/close 2041 (time 1787703745.99 total 30.02 last 71.00) - open/close 2758 (time 1787703756.01 total 40.04 last 71.58) - open/close 3506 (time 1787703766.01 total 50.04 last 74.77) - open/close 4292 (time 1787703776.02 total 60.05 last 78.51) - open/close 4985 (time 1787703786.03 total 70.05 last 69.28) - open/close 5655 (time 1787703796.04 total 80.07 last 66.89) - open/close 6296 (time 1787703806.06 total 90.09 last 64.00) - open/close 6898 (time 1787703816.06 total 100.09 last 60.20) - open/close 7569 (time 1787703826.06 total 110.09 last 67.07) - open/close 8166 (time 1787703836.07 total 120.10 last 59.66) - open/close 8797 (time 1787703846.09 total 130.12 last 62.97) - open/close 9446 (time 1787703856.10 total 140.13 last 64.81) total: 10000 open/close in 148.77 seconds: 67.22 ops/second llite.lustre-ffff98f2b518f800.statahead_batch_max=0 llite.lustre-ffff98f2b518f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98f2b518f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.776s user 0m0.303s sys 0m14.462s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff98f2b518f800.statahead_batch_max=256 llite.lustre-ffff98f2b518f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff98f2b518f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff98f2b518f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.887s user 0m0.357s sys 0m14.894s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff98f2b518f800.batch_stats= snapshot_time: 1787703996.400113761 secs.nsecs start_time: 1787703930.434019887 secs.nsecs elapsed_time: 65.966093874 secs.nsecs subreqs per batch batches %% cum %% 1: 172 48 48 2: 52 14 62 4: 28 7 70 8: 29 8 78 16: 21 5 84 32: 16 4 89 64: 5 1 90 128: 0 0 90 256: 34 9 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff98f2b518f800.statahead_batch_max=64 llite.lustre-ffff98f2b518f800.statahead_max=128