== sanity test 123ad: Verify batching statahead works correctly ========================================================== 14:36:27 (1790102187) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff90f587330000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff90f587330000.batch_stats=0 llite.lustre-ffff90f587330000.statahead_max=32 llite.lustre-ffff90f587330000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff90f587330000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff90f587330000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 3.32 seconds: 30.11 ops/second llite.lustre-ffff90f587330000.statahead_max=0 101 real 0m2.574s user 0m0.005s sys 0m0.353s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff90f587330000.statahead_max=32 llite.lustre-ffff90f587330000.statahead_max=32 101 real 0m1.337s user 0m0.006s sys 0m0.168s 'ls -l' 100 files with statahead: 1 sec statahead total: 110 statahead wrong: 2 agl total: 105 list_total: 31 fname_total: 78 hit_total: 4268 miss_total: 1291 swrong=2 ewrong=2 - open/close 679 (time 1790102217.92 total 10.03 last 67.70) total: 900 open/close in 12.79 seconds: 70.35 ops/second llite.lustre-ffff90f587330000.statahead_max=0 1001 real 0m26.526s user 0m0.035s sys 0m3.278s 'ls -l' 1000 files without statahead: 27 sec llite.lustre-ffff90f587330000.statahead_max=32 llite.lustre-ffff90f587330000.statahead_max=32 1001 real 0m9.471s user 0m0.031s sys 0m1.376s 'ls -l' 1000 files with statahead: 10 sec statahead total: 111 statahead wrong: 2 agl total: 106 list_total: 32 fname_total: 78 hit_total: 5266 miss_total: 1293 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 24 seconds llite.lustre-ffff90f587330000.statahead_stats= statahead total: 111 statahead wrong: 2 agl total: 106 list_total: 32 fname_total: 78 hit_total: 5266 miss_total: 1293 mdc.lustre-MDT0000-mdc-ffff90f587330000.batch_stats= snapshot_time: 1790102295.871832932 secs.nsecs start_time: 1790102192.188527254 secs.nsecs elapsed_time: 103.683305678 secs.nsecs subreqs per batch batches %% cum %% 1: 4 10 10 2: 0 0 10 4: 1 2 13 8: 2 5 18 16: 24 64 83 32: 6 16 100 mdc.lustre-MDT0001-mdc-ffff90f587330000.batch_stats= snapshot_time: 1790102295.871907078 secs.nsecs start_time: 1790102192.188609507 secs.nsecs elapsed_time: 103.683297571 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 2 5 7 8: 3 7 15 16: 11 27 42 32: 23 57 100 - open/close 668 (time 1790102308.68 total 10.00 last 66.78) - open/close 1439 (time 1790102318.69 total 20.01 last 77.04) - open/close 2249 (time 1790102328.70 total 30.02 last 80.94) - open/close 3060 (time 1790102338.72 total 40.03 last 80.96) - open/close 3907 (time 1790102348.72 total 50.03 last 84.70) - open/close 4805 (time 1790102358.73 total 60.04 last 89.71) - open/close 5487 (time 1790102368.76 total 70.07 last 68.00) - open/close 6124 (time 1790102378.76 total 80.08 last 63.66) - open/close 6788 (time 1790102388.79 total 90.11 last 66.21) - open/close 7425 (time 1790102398.80 total 100.12 last 63.63) - open/close 8225 (time 1790102408.81 total 110.13 last 79.92) - open/close 9034 (time 1790102418.81 total 120.13 last 80.89) - open/close 9811 (time 1790102428.82 total 130.14 last 77.63) total: 10000 open/close in 132.94 seconds: 75.22 ops/second llite.lustre-ffff90f587330000.statahead_batch_max=0 llite.lustre-ffff90f587330000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff90f587330000.stats=clear mdc.lustre-MDT0001-mdc-ffff90f587330000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m9.152s user 0m0.355s sys 0m15.824s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff90f587330000.statahead_batch_max=32 llite.lustre-ffff90f587330000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff90f587330000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff90f587330000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff90f587330000.stats=clear mdc.lustre-MDT0001-mdc-ffff90f587330000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m55.098s user 0m0.317s sys 0m14.065s batched statahead RPC= hit count=9965 mdc.lustre-MDT0000-mdc-ffff90f587330000.batch_stats= snapshot_time: 1790102639.250049895 secs.nsecs start_time: 1790102510.495103803 secs.nsecs elapsed_time: 128.754946092 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff90f587330000.batch_stats= snapshot_time: 1790102639.250115677 secs.nsecs start_time: 1790102510.495197385 secs.nsecs elapsed_time: 128.754918292 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff90f587330000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff90f587330000.batch_stats=0 llite.lustre-ffff90f587330000.statahead_max=2048 llite.lustre-ffff90f587330000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff90f587330000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff90f587330000.batch_stats=0 striped dir -i1 -c2 -H all_char /mnt/lustre/d123ad.sanity total: 100 open/close in 3.55 seconds: 28.16 ops/second llite.lustre-ffff90f587330000.statahead_max=0 101 real 0m3.355s user 0m0.004s sys 0m0.366s 'ls -l' 100 files without statahead: 3 sec llite.lustre-ffff90f587330000.statahead_max=2048 llite.lustre-ffff90f587330000.statahead_max=2048 101 real 0m0.979s user 0m0.007s sys 0m0.220s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10064 miss_total: 36 swrong=0 ewrong=0 - open/close 650 (time 1790102952.36 total 10.00 last 64.99) total: 900 open/close in 14.53 seconds: 61.93 ops/second llite.lustre-ffff90f587330000.statahead_max=0 1001 real 0m24.033s user 0m0.038s sys 0m2.935s 'ls -l' 1000 files without statahead: 24 sec llite.lustre-ffff90f587330000.statahead_max=2048 llite.lustre-ffff90f587330000.statahead_max=2048 1001 real 0m7.142s user 0m0.029s sys 0m1.451s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11063 miss_total: 37 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 23 seconds llite.lustre-ffff90f587330000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11063 miss_total: 37 mdc.lustre-MDT0000-mdc-ffff90f587330000.batch_stats= snapshot_time: 1790103025.493198536 secs.nsecs start_time: 1790102642.194549925 secs.nsecs elapsed_time: 383.298648611 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 14 14 8: 2 28 42 16: 1 14 57 32: 1 14 71 64: 0 0 71 128: 0 0 71 256: 2 28 100 mdc.lustre-MDT0001-mdc-ffff90f587330000.batch_stats= snapshot_time: 1790103025.493516697 secs.nsecs start_time: 1790102642.194678477 secs.nsecs elapsed_time: 383.298838220 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 14 14 4: 1 14 28 8: 2 28 57 16: 0 0 57 32: 0 0 57 64: 1 14 71 128: 0 0 71 256: 2 28 100 - open/close 666 (time 1790103038.21 total 10.01 last 66.57) - open/close 1342 (time 1790103048.22 total 20.01 last 67.54) - open/close 2091 (time 1790103058.23 total 30.02 last 74.88) - open/close 2765 (time 1790103068.24 total 40.03 last 67.32) - open/close 3353 (time 1790103078.25 total 50.04 last 58.73) - open/close 3915 (time 1790103088.25 total 60.04 last 56.18) - open/close 4767 (time 1790103098.26 total 70.05 last 85.13) - open/close 5654 (time 1790103108.28 total 80.07 last 88.57) - open/close 6466 (time 1790103118.28 total 90.07 last 81.20) - open/close 7083 (time 1790103128.28 total 100.07 last 61.68) - open/close 7862 (time 1790103138.28 total 110.08 last 77.86) - open/close 8642 (time 1790103148.29 total 120.08 last 77.96) - open/close 9390 (time 1790103158.30 total 130.09 last 74.73) total: 10000 open/close in 137.87 seconds: 72.53 ops/second llite.lustre-ffff90f587330000.statahead_batch_max=0 llite.lustre-ffff90f587330000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff90f587330000.stats=clear mdc.lustre-MDT0001-mdc-ffff90f587330000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m7.280s user 0m0.319s sys 0m14.532s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff90f587330000.statahead_batch_max=256 llite.lustre-ffff90f587330000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff90f587330000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff90f587330000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff90f587330000.stats=clear mdc.lustre-MDT0001-mdc-ffff90f587330000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m52.981s user 0m0.328s sys 0m14.326s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff90f587330000.batch_stats= snapshot_time: 1790103305.563116216 secs.nsecs start_time: 1790103242.212609203 secs.nsecs elapsed_time: 63.350507013 secs.nsecs subreqs per batch batches %% cum %% 1: 118 42 42 2: 40 14 57 4: 31 11 68 8: 21 7 76 16: 7 2 78 32: 19 6 85 64: 4 1 86 128: 2 0 87 256: 34 12 100 mdc.lustre-MDT0001-mdc-ffff90f587330000.batch_stats= snapshot_time: 1790103305.563191896 secs.nsecs start_time: 1790103242.212693843 secs.nsecs elapsed_time: 63.350498053 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff90f587330000.statahead_batch_max=64 llite.lustre-ffff90f587330000.statahead_max=128