== sanity test 123ad: Verify batching statahead works correctly ========================================================== 10:28:03 (1787840883) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.batch_stats=0 llite.lustre-ffff8f9cc348f800.statahead_max=32 llite.lustre-ffff8f9cc348f800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.batch_stats=0 total: 100 open/close in 2.74 seconds: 36.51 ops/second llite.lustre-ffff8f9cc348f800.statahead_max=0 101 real 0m2.733s user 0m0.005s sys 0m0.330s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff8f9cc348f800.statahead_max=32 llite.lustre-ffff8f9cc348f800.statahead_max=32 101 real 0m1.314s user 0m0.004s sys 0m0.160s 'ls -l' 100 files with statahead: 2 sec statahead total: 140 statahead wrong: 2 agl total: 135 list_total: 31 fname_total: 100 hit_total: 4098 miss_total: 1295 swrong=2 ewrong=2 - open/close 665 (time 1787840912.77 total 10.00 last 66.49) total: 900 open/close in 14.19 seconds: 63.41 ops/second llite.lustre-ffff8f9cc348f800.statahead_max=0 1001 real 0m25.787s user 0m0.041s sys 0m2.938s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff8f9cc348f800.statahead_max=32 llite.lustre-ffff8f9cc348f800.statahead_max=32 1001 real 0m11.038s user 0m0.033s sys 0m1.342s 'ls -l' 1000 files with statahead: 11 sec statahead total: 141 statahead wrong: 2 agl total: 136 list_total: 32 fname_total: 100 hit_total: 5097 miss_total: 1296 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff8f9cc348f800.statahead_stats= statahead total: 141 statahead wrong: 2 agl total: 136 list_total: 32 fname_total: 100 hit_total: 5097 miss_total: 1296 mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.batch_stats= snapshot_time: 1787840993.484338607 secs.nsecs start_time: 1787840888.498517939 secs.nsecs elapsed_time: 104.985820668 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 611 (time 1787841006.85 total 10.01 last 61.05) - open/close 1380 (time 1787841016.87 total 20.02 last 76.77) - open/close 2061 (time 1787841026.87 total 30.03 last 68.05) - open/close 2801 (time 1787841036.88 total 40.03 last 73.99) - open/close 3558 (time 1787841046.90 total 50.06 last 75.52) - open/close 4168 (time 1787841056.92 total 60.08 last 60.86) - open/close 4717 (time 1787841066.94 total 70.09 last 54.83) - open/close 5462 (time 1787841076.94 total 80.10 last 74.49) - open/close 6168 (time 1787841086.94 total 90.10 last 70.58) - open/close 6888 (time 1787841096.95 total 100.10 last 71.96) - open/close 7535 (time 1787841106.98 total 110.14 last 64.47) - open/close 8217 (time 1787841116.98 total 120.14 last 68.18) - open/close 8990 (time 1787841127.00 total 130.16 last 77.17) - open/close 9729 (time 1787841137.01 total 140.17 last 73.84) total: 10000 open/close in 143.62 seconds: 69.63 ops/second llite.lustre-ffff8f9cc348f800.statahead_batch_max=0 llite.lustre-ffff8f9cc348f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.092s user 0m0.316s sys 0m14.878s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8f9cc348f800.statahead_batch_max=32 llite.lustre-ffff8f9cc348f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m39.252s user 0m0.306s sys 0m13.685s batched statahead RPC= hit count=9982 mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.batch_stats= snapshot_time: 1787841327.142475811 secs.nsecs start_time: 1787841215.941193439 secs.nsecs elapsed_time: 111.201282372 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.batch_stats=0 llite.lustre-ffff8f9cc348f800.statahead_max=2048 llite.lustre-ffff8f9cc348f800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.batch_stats=0 total: 100 open/close in 3.47 seconds: 28.83 ops/second llite.lustre-ffff8f9cc348f800.statahead_max=0 101 real 0m2.278s user 0m0.004s sys 0m0.309s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8f9cc348f800.statahead_max=2048 llite.lustre-ffff8f9cc348f800.statahead_max=2048 101 real 0m1.031s user 0m0.005s sys 0m0.168s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10081 miss_total: 19 swrong=0 ewrong=0 - open/close 765 (time 1787841655.71 total 10.00 last 76.48) total: 900 open/close in 11.95 seconds: 75.29 ops/second llite.lustre-ffff8f9cc348f800.statahead_max=0 1001 real 0m22.774s user 0m0.033s sys 0m2.864s 'ls -l' 1000 files without statahead: 23 sec llite.lustre-ffff8f9cc348f800.statahead_max=2048 llite.lustre-ffff8f9cc348f800.statahead_max=2048 1001 real 0m7.574s user 0m0.029s sys 0m1.274s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff8f9cc348f800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11080 miss_total: 20 mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.batch_stats= snapshot_time: 1787841725.224489511 secs.nsecs start_time: 1787841330.183343839 secs.nsecs elapsed_time: 395.041145672 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 0 0 54 128: 1 9 63 256: 4 36 100 - open/close 661 (time 1787841738.08 total 10.01 last 66.02) - open/close 1358 (time 1787841748.09 total 20.02 last 69.65) - open/close 2088 (time 1787841758.10 total 30.04 last 72.88) - open/close 2798 (time 1787841768.12 total 40.05 last 70.92) - open/close 3533 (time 1787841778.12 total 50.06 last 73.44) - open/close 4339 (time 1787841788.13 total 60.06 last 80.59) - open/close 5019 (time 1787841798.13 total 70.06 last 67.97) - open/close 5714 (time 1787841808.15 total 80.09 last 69.34) - open/close 6458 (time 1787841818.16 total 90.10 last 74.31) - open/close 7126 (time 1787841828.17 total 100.10 last 66.76) - open/close 7832 (time 1787841838.18 total 110.11 last 70.55) - open/close 8513 (time 1787841848.19 total 120.12 last 68.04) - open/close 9219 (time 1787841858.21 total 130.14 last 70.44) - open/close 9961 (time 1787841868.21 total 140.14 last 74.18) total: 10000 open/close in 140.76 seconds: 71.04 ops/second llite.lustre-ffff8f9cc348f800.statahead_batch_max=0 llite.lustre-ffff8f9cc348f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m57.947s user 0m0.299s sys 0m15.684s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8f9cc348f800.statahead_batch_max=256 llite.lustre-ffff8f9cc348f800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m53.595s user 0m0.272s sys 0m15.044s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8f9cc348f800.batch_stats= snapshot_time: 1787842000.210891272 secs.nsecs start_time: 1787841935.620388201 secs.nsecs elapsed_time: 64.590503071 secs.nsecs subreqs per batch batches %% cum %% 1: 84 45 45 2: 14 7 53 4: 12 6 59 8: 9 4 64 16: 8 4 69 32: 16 8 77 64: 4 2 79 128: 1 0 80 256: 36 19 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8f9cc348f800.statahead_batch_max=64 llite.lustre-ffff8f9cc348f800.statahead_max=128