== sanity test 123ad: Verify batching statahead works correctly ========================================================== 06:03:11 (1789120991) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff92b227a37000.batch_stats=0 llite.lustre-ffff92b227a37000.statahead_max=32 llite.lustre-ffff92b227a37000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff92b227a37000.batch_stats=0 total: 100 open/close in 3.13 seconds: 31.93 ops/second llite.lustre-ffff92b227a37000.statahead_max=0 101 real 0m2.450s user 0m0.004s sys 0m0.298s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff92b227a37000.statahead_max=32 llite.lustre-ffff92b227a37000.statahead_max=32 101 real 0m1.339s user 0m0.001s sys 0m0.177s 'ls -l' 100 files with statahead: 1 sec statahead total: 103 statahead wrong: 2 agl total: 98 list_total: 30 fname_total: 70 hit_total: 4110 miss_total: 1283 swrong=2 ewrong=2 - open/close 601 (time 1789121019.96 total 10.01 last 60.02) total: 900 open/close in 14.91 seconds: 60.38 ops/second llite.lustre-ffff92b227a37000.statahead_max=0 1001 real 0m24.191s user 0m0.039s sys 0m3.214s 'ls -l' 1000 files without statahead: 25 sec llite.lustre-ffff92b227a37000.statahead_max=32 llite.lustre-ffff92b227a37000.statahead_max=32 1001 real 0m10.074s user 0m0.033s sys 0m1.450s 'ls -l' 1000 files with statahead: 11 sec statahead total: 104 statahead wrong: 2 agl total: 99 list_total: 31 fname_total: 70 hit_total: 5106 miss_total: 1287 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff92b227a37000.statahead_stats= statahead total: 104 statahead wrong: 2 agl total: 99 list_total: 31 fname_total: 70 hit_total: 5106 miss_total: 1287 mdc.lustre-MDT0000-mdc-ffff92b227a37000.batch_stats= snapshot_time: 1789121094.047242934 secs.nsecs start_time: 1789120996.334006485 secs.nsecs elapsed_time: 97.713236449 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 3 7 7 16: 2 5 12 32: 34 87 100 - open/close 578 (time 1789121106.79 total 10.02 last 57.66) - open/close 1134 (time 1789121116.79 total 20.03 last 55.56) - open/close 1652 (time 1789121126.81 total 30.05 last 51.70) - open/close 2247 (time 1789121136.81 total 40.05 last 59.49) - open/close 2847 (time 1789121146.82 total 50.06 last 59.98) - open/close 3426 (time 1789121156.83 total 60.07 last 57.85) - open/close 4034 (time 1789121166.83 total 70.07 last 60.79) - open/close 4634 (time 1789121176.85 total 80.09 last 59.86) - open/close 5211 (time 1789121186.86 total 90.10 last 57.64) - open/close 5804 (time 1789121196.87 total 100.11 last 59.27) - open/close 6431 (time 1789121206.88 total 110.12 last 62.64) - open/close 6979 (time 1789121216.90 total 120.14 last 54.65) - open/close 7573 (time 1789121226.92 total 130.16 last 59.33) - open/close 8178 (time 1789121236.92 total 140.16 last 60.50) - open/close 8763 (time 1789121246.94 total 150.18 last 58.34) - open/close 9340 (time 1789121256.95 total 160.19 last 57.68) - open/close 9913 (time 1789121266.96 total 170.20 last 57.24) total: 10000 open/close in 171.98 seconds: 58.15 ops/second llite.lustre-ffff92b227a37000.statahead_batch_max=0 llite.lustre-ffff92b227a37000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92b227a37000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m4.085s user 0m0.379s sys 0m15.879s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92b227a37000.statahead_batch_max=32 llite.lustre-ffff92b227a37000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92b227a37000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92b227a37000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m52.135s user 0m0.355s sys 0m14.419s batched statahead RPC= hit count=9975 mdc.lustre-MDT0000-mdc-ffff92b227a37000.batch_stats= snapshot_time: 1789121465.988215130 secs.nsecs start_time: 1789121341.957314685 secs.nsecs elapsed_time: 124.030900445 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff92b227a37000.batch_stats=0 llite.lustre-ffff92b227a37000.statahead_max=2048 llite.lustre-ffff92b227a37000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff92b227a37000.batch_stats=0 total: 100 open/close in 3.34 seconds: 29.92 ops/second llite.lustre-ffff92b227a37000.statahead_max=0 101 real 0m2.831s user 0m0.004s sys 0m0.358s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff92b227a37000.statahead_max=2048 llite.lustre-ffff92b227a37000.statahead_max=2048 101 real 0m1.030s user 0m0.004s sys 0m0.126s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10074 miss_total: 26 swrong=0 ewrong=0 - open/close 565 (time 1789121810.04 total 10.03 last 56.36) total: 900 open/close in 16.24 seconds: 55.43 ops/second llite.lustre-ffff92b227a37000.statahead_max=0 1001 real 0m26.419s user 0m0.031s sys 0m3.194s 'ls -l' 1000 files without statahead: 26 sec llite.lustre-ffff92b227a37000.statahead_max=2048 llite.lustre-ffff92b227a37000.statahead_max=2048 1001 real 0m7.663s user 0m0.022s sys 0m1.349s 'ls -l' 1000 files with statahead: 8 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11073 miss_total: 27 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff92b227a37000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11073 miss_total: 27 mdc.lustre-MDT0000-mdc-ffff92b227a37000.batch_stats= snapshot_time: 1789121883.834422357 secs.nsecs start_time: 1789121468.390859053 secs.nsecs elapsed_time: 415.443563304 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 18 18 16: 1 9 27 32: 3 27 54 64: 1 9 63 128: 0 0 63 256: 4 36 100 - open/close 646 (time 1789121896.05 total 10.00 last 64.58) - open/close 1295 (time 1789121906.05 total 20.01 last 64.87) - open/close 1974 (time 1789121916.06 total 30.01 last 67.87) - open/close 2624 (time 1789121926.06 total 40.02 last 64.96) - open/close 3254 (time 1789121936.07 total 50.02 last 62.99) - open/close 3867 (time 1789121946.07 total 60.03 last 61.25) - open/close 4414 (time 1789121956.09 total 70.05 last 54.60) - open/close 4961 (time 1789121966.10 total 80.05 last 54.66) - open/close 5468 (time 1789121976.14 total 90.09 last 50.51) - open/close 5895 (time 1789121986.18 total 100.14 last 42.51) - open/close 6382 (time 1789121996.19 total 110.15 last 48.65) - open/close 7003 (time 1789122006.20 total 120.15 last 62.07) - open/close 7639 (time 1789122016.21 total 130.16 last 63.53) - open/close 8278 (time 1789122026.22 total 140.17 last 63.83) - open/close 8926 (time 1789122036.23 total 150.18 last 64.73) - open/close 9547 (time 1789122046.23 total 160.18 last 62.09) total: 10000 open/close in 167.50 seconds: 59.70 ops/second llite.lustre-ffff92b227a37000.statahead_batch_max=0 llite.lustre-ffff92b227a37000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92b227a37000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m54.909s user 0m0.336s sys 0m15.856s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff92b227a37000.statahead_batch_max=256 llite.lustre-ffff92b227a37000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff92b227a37000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff92b227a37000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m51.608s user 0m0.271s sys 0m14.244s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff92b227a37000.batch_stats= snapshot_time: 1789122177.288412317 secs.nsecs start_time: 1789122115.888342277 secs.nsecs elapsed_time: 61.400070040 secs.nsecs subreqs per batch batches %% cum %% 1: 73 38 38 2: 23 12 51 4: 10 5 56 8: 14 7 63 16: 13 6 70 32: 12 6 77 64: 7 3 80 128: 1 0 81 256: 35 18 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff92b227a37000.statahead_batch_max=64 llite.lustre-ffff92b227a37000.statahead_max=128