== sanity test 123ad: Verify batching statahead works correctly ========================================================== 17:32:13 (1777584733) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff962782fda800.batch_stats=0 llite.lustre-ffff962782fda800.statahead_max=32 llite.lustre-ffff962782fda800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff962782fda800.batch_stats=0 total: 100 open/close in 2.70 seconds: 37.06 ops/second llite.lustre-ffff962782fda800.statahead_max=0 101 real 0m2.268s user 0m0.005s sys 0m0.356s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff962782fda800.statahead_max=32 llite.lustre-ffff962782fda800.statahead_max=32 101 real 0m1.046s user 0m0.004s sys 0m0.202s 'ls -l' 100 files with statahead: 1 sec statahead total: 133 statahead wrong: 2 agl total: 127 list_total: 31 fname_total: 81 hit_total: 3973 miss_total: 1503 swrong=2 ewrong=2 - open/close 664 (time 1777584760.67 total 10.01 last 66.34) total: 900 open/close in 13.61 seconds: 66.15 ops/second llite.lustre-ffff962782fda800.statahead_max=0 1001 real 0m26.344s user 0m0.046s sys 0m3.293s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff962782fda800.statahead_max=32 llite.lustre-ffff962782fda800.statahead_max=32 1001 real 0m11.323s user 0m0.026s sys 0m1.602s 'ls -l' 1000 files with statahead: 11 sec statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 32 fname_total: 81 hit_total: 4968 miss_total: 1508 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 20 seconds llite.lustre-ffff962782fda800.statahead_stats= statahead total: 134 statahead wrong: 2 agl total: 128 list_total: 32 fname_total: 81 hit_total: 4968 miss_total: 1508 mdc.lustre-MDT0000-mdc-ffff962782fda800.batch_stats= snapshot_time: 1777584837.364610977 secs.nsecs start_time: 1777584738.163907419 secs.nsecs elapsed_time: 99.200703558 secs.nsecs subreqs per batch batches % cum % 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 2 5 7 16: 3 7 15 32: 34 85 100 - open/close 729 (time 1777584850.46 total 10.01 last 72.85) - open/close 1427 (time 1777584860.47 total 20.01 last 69.74) - open/close 2107 (time 1777584870.48 total 30.02 last 67.96) - open/close 2808 (time 1777584880.48 total 40.02 last 70.09) - open/close 3509 (time 1777584890.49 total 50.03 last 70.05) - open/close 4252 (time 1777584900.49 total 60.04 last 74.23) - open/close 4977 (time 1777584910.50 total 70.05 last 72.45) - open/close 5719 (time 1777584920.51 total 80.06 last 74.12) - open/close 6442 (time 1777584930.52 total 90.07 last 72.21) - open/close 7076 (time 1777584940.53 total 100.07 last 63.37) - open/close 7729 (time 1777584950.53 total 110.08 last 65.28) - open/close 8363 (time 1777584960.54 total 120.08 last 63.36) - open/close 9083 (time 1777584970.54 total 130.09 last 71.97) - open/close 9802 (time 1777584980.55 total 140.09 last 71.87) total: 10000 open/close in 143.14 seconds: 69.86 ops/second llite.lustre-ffff962782fda800.statahead_batch_max=0 llite.lustre-ffff962782fda800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff962782fda800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m56.715s user 0m0.330s sys 0m16.372s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff962782fda800.statahead_batch_max=32 llite.lustre-ffff962782fda800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff962782fda800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff962782fda800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m27.465s user 0m0.264s sys 0m15.032s batched statahead RPC= hit count=9979 mdc.lustre-MDT0000-mdc-ffff962782fda800.batch_stats= snapshot_time: 1777585151.878701656 secs.nsecs start_time: 1777585050.611687454 secs.nsecs elapsed_time: 101.267014202 secs.nsecs subreqs per batch batches % cum % 1: 1 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff962782fda800.batch_stats=0 llite.lustre-ffff962782fda800.statahead_max=2048 llite.lustre-ffff962782fda800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff962782fda800.batch_stats=0 total: 100 open/close in 2.24 seconds: 44.70 ops/second llite.lustre-ffff962782fda800.statahead_max=0 101 real 0m1.814s user 0m0.007s sys 0m0.312s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff962782fda800.statahead_max=2048 llite.lustre-ffff962782fda800.statahead_max=2048 101 real 0m0.773s user 0m0.004s sys 0m0.197s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10078 miss_total: 22 swrong=0 ewrong=0 - open/close 787 (time 1777585437.68 total 10.01 last 78.60) total: 900 open/close in 11.74 seconds: 76.65 ops/second llite.lustre-ffff962782fda800.statahead_max=0 1001 real 0m19.248s user 0m0.026s sys 0m2.937s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff962782fda800.statahead_max=2048 llite.lustre-ffff962782fda800.statahead_max=2048 1001 real 0m6.701s user 0m0.033s sys 0m1.636s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 16 seconds llite.lustre-ffff962782fda800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11077 miss_total: 23 mdc.lustre-MDT0000-mdc-ffff962782fda800.batch_stats= snapshot_time: 1777585492.201394393 secs.nsecs start_time: 1777585153.708158379 secs.nsecs elapsed_time: 338.493236014 secs.nsecs subreqs per batch batches % cum % 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 795 (time 1777585504.15 total 10.00 last 79.48) - open/close 1545 (time 1777585514.15 total 20.01 last 74.98) - open/close 2373 (time 1777585524.15 total 30.01 last 82.79) - open/close 3170 (time 1777585534.16 total 40.01 last 79.65) - open/close 3911 (time 1777585544.16 total 50.02 last 74.06) - open/close 4640 (time 1777585554.17 total 60.02 last 72.86) - open/close 5374 (time 1777585564.18 total 70.04 last 73.30) - open/close 6152 (time 1777585574.19 total 80.04 last 77.77) - open/close 6907 (time 1777585584.20 total 90.05 last 75.39) - open/close 7648 (time 1777585594.20 total 100.06 last 74.07) - open/close 8401 (time 1777585604.23 total 110.08 last 75.14) - open/close 9113 (time 1777585614.23 total 120.09 last 71.16) - open/close 9806 (time 1777585624.24 total 130.09 last 69.24) total: 10000 open/close in 132.63 seconds: 75.40 ops/second llite.lustre-ffff962782fda800.statahead_batch_max=0 llite.lustre-ffff962782fda800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff962782fda800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m43.846s user 0m0.339s sys 0m18.751s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff962782fda800.statahead_batch_max=256 llite.lustre-ffff962782fda800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff962782fda800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff962782fda800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m43.235s user 0m0.256s sys 0m16.711s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff962782fda800.batch_stats= snapshot_time: 1777585733.992876463 secs.nsecs start_time: 1777585679.301778655 secs.nsecs elapsed_time: 54.691097808 secs.nsecs subreqs per batch batches % cum % 1: 226 53 53 2: 55 13 66 4: 48 11 78 8: 22 5 83 16: 18 4 87 32: 15 3 91 64: 1 0 91 128: 2 0 91 256: 34 8 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff962782fda800.statahead_batch_max=64 llite.lustre-ffff962782fda800.statahead_max=128