== sanity test 123ad: Verify batching statahead works correctly ========================================================== 14:23:10 (1787854990) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.batch_stats=0 llite.lustre-ffff8c0c8091c000.statahead_max=32 llite.lustre-ffff8c0c8091c000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.batch_stats=0 total: 100 open/close in 2.90 seconds: 34.51 ops/second llite.lustre-ffff8c0c8091c000.statahead_max=0 101 real 0m2.177s user 0m0.006s sys 0m0.338s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8c0c8091c000.statahead_max=32 llite.lustre-ffff8c0c8091c000.statahead_max=32 101 real 0m1.025s user 0m0.005s sys 0m0.167s 'ls -l' 100 files with statahead: 1 sec statahead total: 159 statahead wrong: 2 agl total: 152 list_total: 30 fname_total: 129 hit_total: 3953 miss_total: 1301 swrong=2 ewrong=2 - open/close 715 (time 1787855020.30 total 10.01 last 71.45) total: 900 open/close in 12.69 seconds: 70.90 ops/second llite.lustre-ffff8c0c8091c000.statahead_max=0 1001 real 0m19.452s user 0m0.034s sys 0m3.018s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff8c0c8091c000.statahead_max=32 llite.lustre-ffff8c0c8091c000.statahead_max=32 1001 real 0m10.067s user 0m0.036s sys 0m1.319s 'ls -l' 1000 files with statahead: 11 sec statahead total: 160 statahead wrong: 2 agl total: 153 list_total: 31 fname_total: 129 hit_total: 4951 miss_total: 1303 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff8c0c8091c000.statahead_stats= statahead total: 160 statahead wrong: 2 agl total: 153 list_total: 31 fname_total: 129 hit_total: 4951 miss_total: 1303 mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.batch_stats= snapshot_time: 1787855089.983358190 secs.nsecs start_time: 1787854996.899745993 secs.nsecs elapsed_time: 93.083612197 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 2 2 8: 2 5 7 16: 2 5 12 32: 34 87 100 - open/close 680 (time 1787855102.81 total 10.00 last 67.97) - open/close 1404 (time 1787855112.81 total 20.01 last 72.34) - open/close 2058 (time 1787855122.82 total 30.02 last 65.34) - open/close 2819 (time 1787855132.83 total 40.03 last 76.07) - open/close 3598 (time 1787855142.83 total 50.03 last 77.87) - open/close 4349 (time 1787855152.83 total 60.03 last 75.07) - open/close 5062 (time 1787855162.84 total 70.04 last 71.24) - open/close 5723 (time 1787855172.84 total 80.04 last 66.10) - open/close 6275 (time 1787855182.87 total 90.06 last 55.08) - open/close 6738 (time 1787855192.87 total 100.07 last 46.29) - open/close 7234 (time 1787855202.87 total 110.07 last 49.57) - open/close 7842 (time 1787855212.88 total 120.07 last 60.79) - open/close 8496 (time 1787855222.88 total 130.08 last 65.39) - open/close 9108 (time 1787855232.90 total 140.10 last 61.08) - open/close 9705 (time 1787855242.95 total 150.15 last 59.41) total: 10000 open/close in 154.74 seconds: 64.62 ops/second llite.lustre-ffff8c0c8091c000.statahead_batch_max=0 llite.lustre-ffff8c0c8091c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m10.665s user 0m0.350s sys 0m14.941s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8c0c8091c000.statahead_batch_max=32 llite.lustre-ffff8c0c8091c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m57.014s user 0m0.350s sys 0m14.554s batched statahead RPC= hit count=9973 mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.batch_stats= snapshot_time: 1787855457.268912986 secs.nsecs start_time: 1787855327.877648131 secs.nsecs elapsed_time: 129.391264855 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.batch_stats=0 llite.lustre-ffff8c0c8091c000.statahead_max=2048 llite.lustre-ffff8c0c8091c000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.batch_stats=0 total: 100 open/close in 3.09 seconds: 32.33 ops/second llite.lustre-ffff8c0c8091c000.statahead_max=0 101 real 0m2.447s user 0m0.005s sys 0m0.374s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff8c0c8091c000.statahead_max=2048 llite.lustre-ffff8c0c8091c000.statahead_max=2048 101 real 0m1.347s user 0m0.004s sys 0m0.161s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10072 miss_total: 28 swrong=0 ewrong=0 - open/close 679 (time 1787855794.83 total 10.01 last 67.85) total: 900 open/close in 13.46 seconds: 66.88 ops/second llite.lustre-ffff8c0c8091c000.statahead_max=0 1001 real 0m22.099s user 0m0.041s sys 0m3.448s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff8c0c8091c000.statahead_max=2048 llite.lustre-ffff8c0c8091c000.statahead_max=2048 1001 real 0m7.282s user 0m0.029s sys 0m1.341s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11071 miss_total: 29 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 30 seconds llite.lustre-ffff8c0c8091c000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11071 miss_total: 29 mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.batch_stats= snapshot_time: 1787855871.515402938 secs.nsecs start_time: 1787855460.339630728 secs.nsecs elapsed_time: 411.175772210 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 653 (time 1787855885.07 total 10.01 last 65.22) - open/close 1276 (time 1787855895.08 total 20.02 last 62.24) - open/close 1912 (time 1787855905.09 total 30.03 last 63.55) - open/close 2560 (time 1787855915.11 total 40.04 last 64.70) - open/close 3197 (time 1787855925.12 total 50.05 last 63.64) - open/close 3881 (time 1787855935.12 total 60.06 last 68.37) - open/close 4493 (time 1787855945.12 total 70.06 last 61.17) - open/close 5172 (time 1787855955.13 total 80.06 last 67.88) - open/close 5854 (time 1787855965.13 total 90.07 last 68.14) - open/close 6510 (time 1787855975.16 total 100.09 last 65.47) - open/close 7148 (time 1787855985.17 total 110.11 last 63.71) - open/close 7788 (time 1787855995.18 total 120.11 last 63.97) - open/close 8419 (time 1787856005.18 total 130.12 last 63.07) - open/close 9035 (time 1787856015.18 total 140.12 last 61.60) - open/close 9672 (time 1787856025.19 total 150.13 last 63.62) total: 10000 open/close in 154.93 seconds: 64.55 ops/second llite.lustre-ffff8c0c8091c000.statahead_batch_max=0 llite.lustre-ffff8c0c8091c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.506s user 0m0.375s sys 0m14.850s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff8c0c8091c000.statahead_batch_max=256 llite.lustre-ffff8c0c8091c000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.382s user 0m0.371s sys 0m15.252s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff8c0c8091c000.batch_stats= snapshot_time: 1787856173.213347911 secs.nsecs start_time: 1787856099.853704238 secs.nsecs elapsed_time: 73.359643673 secs.nsecs subreqs per batch batches %% cum %% 1: 140 40 40 2: 72 20 60 4: 42 12 72 8: 25 7 80 16: 16 4 84 32: 14 4 88 64: 5 1 90 128: 0 0 90 256: 34 9 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff8c0c8091c000.statahead_batch_max=64 llite.lustre-ffff8c0c8091c000.statahead_max=128