== sanity test 123ad: Verify batching statahead works correctly ========================================================== 07:19:50 (1789384790) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9b4491009000.batch_stats=0 llite.lustre-ffff9b4491009000.statahead_max=32 llite.lustre-ffff9b4491009000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9b4491009000.batch_stats=0 total: 100 open/close in 3.07 seconds: 32.54 ops/second llite.lustre-ffff9b4491009000.statahead_max=0 101 real 0m2.519s user 0m0.006s sys 0m0.337s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9b4491009000.statahead_max=32 llite.lustre-ffff9b4491009000.statahead_max=32 101 real 0m1.258s user 0m0.005s sys 0m0.180s 'ls -l' 100 files with statahead: 1 sec statahead total: 144 statahead wrong: 2 agl total: 138 list_total: 30 fname_total: 112 hit_total: 3958 miss_total: 1516 swrong=2 ewrong=2 - open/close 675 (time 1789384818.92 total 10.02 last 67.35) total: 900 open/close in 13.35 seconds: 67.40 ops/second llite.lustre-ffff9b4491009000.statahead_max=0 1001 real 0m24.177s user 0m0.031s sys 0m2.888s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff9b4491009000.statahead_max=32 llite.lustre-ffff9b4491009000.statahead_max=32 1001 real 0m11.369s user 0m0.031s sys 0m1.313s 'ls -l' 1000 files with statahead: 11 sec statahead total: 145 statahead wrong: 2 agl total: 139 list_total: 31 fname_total: 112 hit_total: 4953 miss_total: 1521 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff9b4491009000.statahead_stats= statahead total: 145 statahead wrong: 2 agl total: 139 list_total: 31 fname_total: 112 hit_total: 4953 miss_total: 1521 mdc.lustre-MDT0000-mdc-ffff9b4491009000.batch_stats= snapshot_time: 1789384896.060657790 secs.nsecs start_time: 1789384795.146303430 secs.nsecs elapsed_time: 100.914354360 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 2 2 8: 2 5 7 16: 2 5 12 32: 34 87 100 - open/close 629 (time 1789384909.16 total 10.01 last 62.82) - open/close 1257 (time 1789384919.16 total 20.02 last 62.78) - open/close 1835 (time 1789384929.17 total 30.02 last 57.76) - open/close 2479 (time 1789384939.18 total 40.04 last 64.31) - open/close 3072 (time 1789384949.19 total 50.05 last 59.24) - open/close 3674 (time 1789384959.20 total 60.05 last 60.15) - open/close 4197 (time 1789384969.24 total 70.10 last 52.07) - open/close 4827 (time 1789384979.25 total 80.11 last 62.94) - open/close 5430 (time 1789384989.26 total 90.11 last 60.28) - open/close 6037 (time 1789384999.26 total 100.11 last 60.70) - open/close 6669 (time 1789385009.26 total 110.12 last 63.16) - open/close 7282 (time 1789385019.27 total 120.12 last 61.26) - open/close 7942 (time 1789385029.28 total 130.14 last 65.91) - open/close 8493 (time 1789385039.29 total 140.15 last 55.05) - open/close 9096 (time 1789385049.30 total 150.15 last 60.27) - open/close 9662 (time 1789385059.31 total 160.16 last 56.53) total: 10000 open/close in 165.48 seconds: 60.43 ops/second llite.lustre-ffff9b4491009000.statahead_batch_max=0 llite.lustre-ffff9b4491009000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b4491009000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m6.861s user 0m0.363s sys 0m14.996s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9b4491009000.statahead_batch_max=32 llite.lustre-ffff9b4491009000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b4491009000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9b4491009000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m50.899s user 0m0.343s sys 0m14.388s batched statahead RPC= hit count=9972 mdc.lustre-MDT0000-mdc-ffff9b4491009000.batch_stats= snapshot_time: 1789385263.419073723 secs.nsecs start_time: 1789385141.743251427 secs.nsecs elapsed_time: 121.675822296 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9b4491009000.batch_stats=0 llite.lustre-ffff9b4491009000.statahead_max=2048 llite.lustre-ffff9b4491009000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9b4491009000.batch_stats=0 total: 100 open/close in 3.48 seconds: 28.71 ops/second llite.lustre-ffff9b4491009000.statahead_max=0 101 real 0m3.067s user 0m0.004s sys 0m0.301s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff9b4491009000.statahead_max=2048 llite.lustre-ffff9b4491009000.statahead_max=2048 101 real 0m0.940s user 0m0.004s sys 0m0.133s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10071 miss_total: 29 swrong=0 ewrong=0 - open/close 511 (time 1789385630.89 total 10.00 last 51.09) total: 900 open/close in 17.24 seconds: 52.19 ops/second llite.lustre-ffff9b4491009000.statahead_max=0 1001 real 0m26.518s user 0m0.030s sys 0m2.873s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff9b4491009000.statahead_max=2048 llite.lustre-ffff9b4491009000.statahead_max=2048 1001 real 0m7.120s user 0m0.036s sys 0m1.363s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11070 miss_total: 30 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 25 seconds llite.lustre-ffff9b4491009000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11070 miss_total: 30 mdc.lustre-MDT0000-mdc-ffff9b4491009000.batch_stats= snapshot_time: 1789385711.480169631 secs.nsecs start_time: 1789385266.053978387 secs.nsecs elapsed_time: 445.426191244 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 587 (time 1789385724.65 total 10.01 last 58.66) - open/close 1255 (time 1789385734.66 total 20.01 last 66.76) - open/close 1861 (time 1789385744.67 total 30.03 last 60.50) - open/close 2528 (time 1789385754.71 total 40.06 last 66.47) - open/close 3128 (time 1789385764.72 total 50.08 last 59.93) - open/close 3721 (time 1789385774.73 total 60.09 last 59.21) - open/close 4346 (time 1789385784.74 total 70.09 last 62.49) - open/close 4985 (time 1789385794.74 total 80.10 last 63.88) - open/close 5556 (time 1789385804.75 total 90.11 last 57.04) - open/close 6165 (time 1789385814.75 total 100.11 last 60.87) - open/close 6708 (time 1789385824.78 total 110.13 last 54.19) - open/close 7378 (time 1789385834.78 total 120.14 last 66.96) - open/close 7924 (time 1789385844.78 total 130.14 last 54.60) - open/close 8422 (time 1789385854.79 total 140.15 last 49.74) - open/close 8969 (time 1789385864.80 total 150.16 last 54.66) - open/close 9505 (time 1789385874.81 total 160.17 last 53.53) total: 10000 open/close in 168.89 seconds: 59.21 ops/second llite.lustre-ffff9b4491009000.statahead_batch_max=0 llite.lustre-ffff9b4491009000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b4491009000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.158s user 0m0.367s sys 0m15.647s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9b4491009000.statahead_batch_max=256 llite.lustre-ffff9b4491009000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9b4491009000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9b4491009000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m1.718s user 0m0.329s sys 0m15.373s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9b4491009000.batch_stats= snapshot_time: 1789386033.345334304 secs.nsecs start_time: 1789385959.697066506 secs.nsecs elapsed_time: 73.648267798 secs.nsecs subreqs per batch batches %% cum %% 1: 121 43 43 2: 40 14 57 4: 29 10 68 8: 20 7 75 16: 10 3 79 32: 18 6 85 64: 4 1 87 128: 1 0 87 256: 35 12 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9b4491009000.statahead_batch_max=64 llite.lustre-ffff9b4491009000.statahead_max=128