== sanity test 123ad: Verify batching statahead works correctly ========================================================== 08:30:59 (1787747459) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9b6512d81000.batch_stats=0 llite.lustre-ffff9b6512d81000.statahead_max=32 llite.lustre-ffff9b6512d81000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9b6512d81000.batch_stats=0 total: 100 open/close in 2.50 seconds: 40.00 ops/second llite.lustre-ffff9b6512d81000.statahead_max=0 101 real 0m2.453s user 0m0.001s sys 0m0.312s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9b6512d81000.statahead_max=32 llite.lustre-ffff9b6512d81000.statahead_max=32 101 real 0m1.160s user 0m0.007s sys 0m0.154s 'ls -l' 100 files with statahead: 1 sec statahead total: 120 statahead wrong: 2 agl total: 113 list_total: 30 fname_total: 85 hit_total: 3971 miss_total: 1554 swrong=2 ewrong=2 - open/close 670 (time 1787747487.24 total 10.01 last 66.96) total: 900 open/close in 14.14 seconds: 63.66 ops/second llite.lustre-ffff9b6512d81000.statahead_max=0 1001 real 0m28.209s user 0m0.031s sys 0m2.810s 'ls -l' 1000 files without statahead: 28 sec llite.lustre-ffff9b6512d81000.statahead_max=32 llite.lustre-ffff9b6512d81000.statahead_max=32 1001 real 0m15.176s user 0m0.029s sys 0m1.335s 'ls -l' 1000 files with statahead: 16 sec statahead total: 121 statahead wrong: 2 agl total: 114 list_total: 31 fname_total: 85 hit_total: 4966 miss_total: 1559 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 30 seconds llite.lustre-ffff9b6512d81000.statahead_stats= statahead total: 121 statahead wrong: 2 agl total: 114 list_total: 31 fname_total: 85 hit_total: 4966 miss_total: 1559 mdc.lustre-MDT0000-mdc-ffff9b6512d81000.batch_stats= snapshot_time: 1787747580.563069278 secs.nsecs start_time: 1787747464.025974289 secs.nsecs elapsed_time: 116.537094989 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 5 5 16: 3 7 12 32: 34 87 100 - open/close 629 (time 1787747593.38 total 10.00 last 62.89) - open/close 1259 (time 1787747603.39 total 20.01 last 62.92) - open/close 1937 (time 1787747613.39 total 30.02 last 67.78) - open/close 2603 (time 1787747623.40 total 40.02 last 66.57) - open/close 3293 (time 1787747633.40 total 50.03 last 68.97) - open/close 4014 (time 1787747643.43 total 60.05 last 71.93) - open/close 4732 (time 1787747653.43 total 70.06 last 71.75) - open/close 5416 (time 1787747663.45 total 80.07 last 68.28) - open/close 6078 (time 1787747673.45 total 90.08 last 66.17) - open/close 6732 (time 1787747683.46 total 100.08 last 65.38) - open/close 7371 (time 1787747693.47 total 110.09 last 63.84) - open/close 8017 (time 1787747703.49 total 120.11 last 64.47) - open/close 8694 (time 1787747713.49 total 130.11 last 67.68) - open/close 9350 (time 1787747723.49 total 140.12 last 65.57) - open/close 9960 (time 1787747733.50 total 150.13 last 60.95) total: 10000 open/close in 151.04 seconds: 66.21 ops/second llite.lustre-ffff9b6512d81000.statahead_batch_max=0 llite.lustre-ffff9b6512d81000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b6512d81000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m5.098s user 0m0.263s sys 0m13.892s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9b6512d81000.statahead_batch_max=32 llite.lustre-ffff9b6512d81000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b6512d81000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9b6512d81000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m46.157s user 0m0.288s sys 0m12.928s batched statahead RPC= hit count=9970 mdc.lustre-MDT0000-mdc-ffff9b6512d81000.batch_stats= snapshot_time: 1787747930.413026346 secs.nsecs start_time: 1787747812.149922867 secs.nsecs elapsed_time: 118.263103479 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9b6512d81000.batch_stats=0 llite.lustre-ffff9b6512d81000.statahead_max=2048 llite.lustre-ffff9b6512d81000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9b6512d81000.batch_stats=0 total: 100 open/close in 2.81 seconds: 35.64 ops/second llite.lustre-ffff9b6512d81000.statahead_max=0 101 real 0m2.336s user 0m0.006s sys 0m0.300s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9b6512d81000.statahead_max=2048 llite.lustre-ffff9b6512d81000.statahead_max=2048 101 real 0m0.799s user 0m0.004s sys 0m0.161s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10069 miss_total: 31 swrong=0 ewrong=0 - open/close 766 (time 1787748256.27 total 10.01 last 76.55) total: 900 open/close in 12.10 seconds: 74.38 ops/second llite.lustre-ffff9b6512d81000.statahead_max=0 1001 real 0m23.205s user 0m0.024s sys 0m2.813s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff9b6512d81000.statahead_max=2048 llite.lustre-ffff9b6512d81000.statahead_max=2048 1001 real 0m7.395s user 0m0.028s sys 0m1.243s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11068 miss_total: 32 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff9b6512d81000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11068 miss_total: 32 mdc.lustre-MDT0000-mdc-ffff9b6512d81000.batch_stats= snapshot_time: 1787748327.347535881 secs.nsecs start_time: 1787747932.949691285 secs.nsecs elapsed_time: 394.397844596 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 600 (time 1787748340.04 total 10.01 last 59.95) - open/close 1267 (time 1787748350.05 total 20.01 last 66.69) - open/close 1932 (time 1787748360.05 total 30.02 last 66.46) - open/close 2706 (time 1787748370.07 total 40.04 last 77.25) - open/close 3419 (time 1787748380.08 total 50.04 last 71.26) - open/close 4191 (time 1787748390.10 total 60.06 last 77.03) - open/close 4954 (time 1787748400.10 total 70.07 last 76.26) - open/close 5620 (time 1787748410.13 total 80.09 last 66.46) - open/close 6313 (time 1787748420.13 total 90.10 last 69.26) - open/close 7049 (time 1787748430.14 total 100.11 last 73.54) - open/close 7833 (time 1787748440.15 total 110.11 last 78.36) - open/close 8500 (time 1787748450.15 total 120.11 last 66.68) - open/close 9176 (time 1787748460.15 total 130.12 last 67.58) - open/close 9973 (time 1787748470.16 total 140.13 last 79.61) total: 10000 open/close in 140.46 seconds: 71.19 ops/second llite.lustre-ffff9b6512d81000.statahead_batch_max=0 llite.lustre-ffff9b6512d81000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b6512d81000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m11.923s user 0m0.305s sys 0m13.820s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9b6512d81000.statahead_batch_max=256 llite.lustre-ffff9b6512d81000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b6512d81000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9b6512d81000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.852s user 0m0.293s sys 0m13.736s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9b6512d81000.batch_stats= snapshot_time: 1787748612.632510778 secs.nsecs start_time: 1787748552.113736050 secs.nsecs elapsed_time: 60.518774728 secs.nsecs subreqs per batch batches %% cum %% 1: 142 44 44 2: 49 15 59 4: 37 11 71 8: 22 6 78 16: 13 4 82 32: 16 5 87 64: 6 1 89 128: 0 0 89 256: 34 10 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9b6512d81000.statahead_batch_max=64 llite.lustre-ffff9b6512d81000.statahead_max=128