== sanity test 123ad: Verify batching statahead works correctly ========================================================== 22:47:08 (1789613228) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff964b90b2a800.batch_stats=0 llite.lustre-ffff964b90b2a800.statahead_max=32 llite.lustre-ffff964b90b2a800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff964b90b2a800.batch_stats=0 total: 100 open/close in 4.09 seconds: 24.44 ops/second llite.lustre-ffff964b90b2a800.statahead_max=0 101 real 0m2.314s user 0m0.004s sys 0m0.357s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff964b90b2a800.statahead_max=32 llite.lustre-ffff964b90b2a800.statahead_max=32 101 real 0m1.347s user 0m0.008s sys 0m0.214s 'ls -l' 100 files with statahead: 1 sec statahead total: 105 statahead wrong: 2 agl total: 98 list_total: 29 fname_total: 65 hit_total: 3968 miss_total: 1607 swrong=2 ewrong=2 - open/close 610 (time 1789613260.02 total 10.00 last 60.98) total: 900 open/close in 14.22 seconds: 63.28 ops/second llite.lustre-ffff964b90b2a800.statahead_max=0 1001 real 0m25.525s user 0m0.052s sys 0m3.253s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff964b90b2a800.statahead_max=32 llite.lustre-ffff964b90b2a800.statahead_max=32 1001 real 0m12.314s user 0m0.033s sys 0m1.438s 'ls -l' 1000 files with statahead: 12 sec statahead total: 106 statahead wrong: 2 agl total: 99 list_total: 30 fname_total: 65 hit_total: 4965 miss_total: 1610 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 26 seconds llite.lustre-ffff964b90b2a800.statahead_stats= statahead total: 106 statahead wrong: 2 agl total: 99 list_total: 30 fname_total: 65 hit_total: 4965 miss_total: 1610 mdc.lustre-MDT0000-mdc-ffff964b90b2a800.batch_stats= snapshot_time: 1789613344.279519519 secs.nsecs start_time: 1789613233.835164002 secs.nsecs elapsed_time: 110.444355517 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 553 (time 1789613357.31 total 10.00 last 55.30) - open/close 1137 (time 1789613367.33 total 20.02 last 58.28) - open/close 1728 (time 1789613377.34 total 30.03 last 59.04) - open/close 2270 (time 1789613387.35 total 40.04 last 54.14) - open/close 2757 (time 1789613397.35 total 50.04 last 48.69) - open/close 3276 (time 1789613407.37 total 60.06 last 51.79) - open/close 3832 (time 1789613417.37 total 70.06 last 55.60) - open/close 4293 (time 1789613427.37 total 80.07 last 46.09) - open/close 5009 (time 1789613437.38 total 90.07 last 71.58) - open/close 5524 (time 1789613447.39 total 100.09 last 51.41) - open/close 6051 (time 1789613457.40 total 110.09 last 52.66) - open/close 6645 (time 1789613467.40 total 120.10 last 59.38) - open/close 7192 (time 1789613477.42 total 130.11 last 54.61) - open/close 7643 (time 1789613487.43 total 140.13 last 45.04) - open/close 8083 (time 1789613497.44 total 150.14 last 43.96) - open/close 8541 (time 1789613507.45 total 160.14 last 45.76) - open/close 9031 (time 1789613517.46 total 170.15 last 48.96) - open/close 9615 (time 1789613527.46 total 180.16 last 58.38) total: 10000 open/close in 186.51 seconds: 53.62 ops/second llite.lustre-ffff964b90b2a800.statahead_batch_max=0 llite.lustre-ffff964b90b2a800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff964b90b2a800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m19.287s user 0m0.371s sys 0m15.448s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff964b90b2a800.statahead_batch_max=32 llite.lustre-ffff964b90b2a800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff964b90b2a800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff964b90b2a800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m50.642s user 0m0.366s sys 0m14.560s batched statahead RPC= hit count=9976 mdc.lustre-MDT0000-mdc-ffff964b90b2a800.batch_stats= snapshot_time: 1789613745.681050234 secs.nsecs start_time: 1789613622.761206487 secs.nsecs elapsed_time: 122.919843747 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff964b90b2a800.batch_stats=0 llite.lustre-ffff964b90b2a800.statahead_max=2048 llite.lustre-ffff964b90b2a800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff964b90b2a800.batch_stats=0 total: 100 open/close in 2.82 seconds: 35.44 ops/second llite.lustre-ffff964b90b2a800.statahead_max=0 101 real 0m2.015s user 0m0.004s sys 0m0.369s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff964b90b2a800.statahead_max=2048 llite.lustre-ffff964b90b2a800.statahead_max=2048 101 real 0m1.168s user 0m0.007s sys 0m0.183s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10075 miss_total: 25 swrong=0 ewrong=0 - open/close 606 (time 1789614089.28 total 10.00 last 60.59) total: 900 open/close in 15.09 seconds: 59.64 ops/second llite.lustre-ffff964b90b2a800.statahead_max=0 1001 real 0m24.189s user 0m0.040s sys 0m2.904s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff964b90b2a800.statahead_max=2048 llite.lustre-ffff964b90b2a800.statahead_max=2048 1001 real 0m7.801s user 0m0.030s sys 0m1.246s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff964b90b2a800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11074 miss_total: 26 mdc.lustre-MDT0000-mdc-ffff964b90b2a800.batch_stats= snapshot_time: 1789614161.666280375 secs.nsecs start_time: 1789613748.460627659 secs.nsecs elapsed_time: 413.205652716 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 2 18 36 32: 2 18 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 590 (time 1789614174.46 total 10.03 last 58.80) - open/close 1264 (time 1789614184.46 total 20.04 last 67.35) - open/close 1906 (time 1789614194.48 total 30.05 last 64.13) - open/close 2528 (time 1789614204.48 total 40.06 last 62.16) - open/close 3081 (time 1789614214.48 total 50.06 last 55.28) - open/close 3654 (time 1789614224.49 total 60.06 last 57.29) - open/close 4198 (time 1789614234.49 total 70.07 last 54.40) - open/close 4627 (time 1789614244.52 total 80.10 last 42.74) - open/close 5164 (time 1789614254.53 total 90.11 last 53.65) - open/close 5751 (time 1789614264.54 total 100.12 last 58.64) - open/close 6371 (time 1789614274.55 total 110.12 last 61.98) - open/close 6972 (time 1789614284.56 total 120.14 last 60.01) - open/close 7610 (time 1789614294.56 total 130.14 last 63.79) - open/close 8225 (time 1789614304.58 total 140.16 last 61.38) - open/close 8851 (time 1789614314.59 total 150.17 last 62.57) - open/close 9397 (time 1789614324.59 total 160.17 last 54.58) total: 10000 open/close in 168.87 seconds: 59.22 ops/second llite.lustre-ffff964b90b2a800.statahead_batch_max=0 llite.lustre-ffff964b90b2a800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff964b90b2a800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.047s user 0m0.363s sys 0m14.486s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff964b90b2a800.statahead_batch_max=256 llite.lustre-ffff964b90b2a800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff964b90b2a800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff964b90b2a800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.167s user 0m0.333s sys 0m14.258s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff964b90b2a800.batch_stats= snapshot_time: 1789614490.126421098 secs.nsecs start_time: 1789614417.452506993 secs.nsecs elapsed_time: 72.673914105 secs.nsecs subreqs per batch batches %% cum %% 1: 93 40 40 2: 31 13 53 4: 23 9 63 8: 14 6 69 16: 16 6 76 32: 13 5 82 64: 6 2 84 128: 0 0 84 256: 35 15 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff964b90b2a800.statahead_batch_max=64 llite.lustre-ffff964b90b2a800.statahead_max=128