== sanity test 123ad: Verify batching statahead works correctly ========================================================== 22:38:59 (1781663939) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.batch_stats=0 llite.lustre-ffff9ba903dfd000.statahead_max=32 llite.lustre-ffff9ba903dfd000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 0.87 seconds: 114.90 ops/second llite.lustre-ffff9ba903dfd000.statahead_max=0 101 real 0m1.057s user 0m0.005s sys 0m0.392s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff9ba903dfd000.statahead_max=32 llite.lustre-ffff9ba903dfd000.statahead_max=32 101 real 0m0.491s user 0m0.003s sys 0m0.222s 'ls -l' 100 files with statahead: 1 sec statahead total: 60 statahead wrong: 2 agl total: 56 list_total: 25 fname_total: 35 hit_total: 14257 miss_total: 718 swrong=2 ewrong=2 total: 900 open/close in 3.20 seconds: 280.98 ops/second llite.lustre-ffff9ba903dfd000.statahead_max=0 1001 real 0m8.458s user 0m0.022s sys 0m3.225s 'ls -l' 1000 files without statahead: 8 sec llite.lustre-ffff9ba903dfd000.statahead_max=32 llite.lustre-ffff9ba903dfd000.statahead_max=32 1001 real 0m5.211s user 0m0.019s sys 0m2.288s 'ls -l' 1000 files with statahead: 5 sec statahead total: 61 statahead wrong: 2 agl total: 57 list_total: 26 fname_total: 35 hit_total: 15256 miss_total: 719 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 7 seconds llite.lustre-ffff9ba903dfd000.statahead_stats= statahead total: 61 statahead wrong: 2 agl total: 57 list_total: 26 fname_total: 35 hit_total: 15256 miss_total: 719 mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.batch_stats= snapshot_time: 1781663975.141141721 secs.nsecs start_time: 1781663941.684917968 secs.nsecs elapsed_time: 33.456223753 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 2 5 5 4: 2 5 10 8: 2 5 15 16: 19 48 64 32: 14 35 100 mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.batch_stats= snapshot_time: 1781663975.141181446 secs.nsecs start_time: 1781663941.684992314 secs.nsecs elapsed_time: 33.456189132 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 2 2 4: 2 5 7 8: 4 10 17 16: 17 43 61 32: 15 38 100 - open/close 3110 (time 1781663986.36 total 10.00 last 310.89) - open/close 7048 (time 1781663996.36 total 20.00 last 393.76) total: 10000 open/close in 26.76 seconds: 373.73 ops/second llite.lustre-ffff9ba903dfd000.statahead_batch_max=0 llite.lustre-ffff9ba903dfd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.stats=clear mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m52.050s user 0m0.165s sys 0m49.266s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ba903dfd000.statahead_batch_max=32 llite.lustre-ffff9ba903dfd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.stats=clear mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m38.482s user 0m0.141s sys 0m22.408s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.batch_stats= snapshot_time: 1781664103.392132763 secs.nsecs start_time: 1781664058.517396323 secs.nsecs elapsed_time: 44.874736440 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.batch_stats= snapshot_time: 1781664103.392171783 secs.nsecs start_time: 1781664058.517476463 secs.nsecs elapsed_time: 44.874695320 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.batch_stats=0 llite.lustre-ffff9ba903dfd000.statahead_max=2048 llite.lustre-ffff9ba903dfd000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 0.70 seconds: 143.84 ops/second llite.lustre-ffff9ba903dfd000.statahead_max=0 101 real 0m0.613s user 0m0.001s sys 0m0.241s 'ls -l' 100 files without statahead: 0 sec llite.lustre-ffff9ba903dfd000.statahead_max=2048 llite.lustre-ffff9ba903dfd000.statahead_max=2048 101 real 0m0.390s user 0m0.002s sys 0m0.254s 'ls -l' 100 files with statahead: 0 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 swrong=0 ewrong=0 total: 900 open/close in 1.93 seconds: 467.27 ops/second llite.lustre-ffff9ba903dfd000.statahead_max=0 1001 real 0m5.866s user 0m0.014s sys 0m2.074s 'ls -l' 1000 files without statahead: 6 sec llite.lustre-ffff9ba903dfd000.statahead_max=2048 llite.lustre-ffff9ba903dfd000.statahead_max=2048 1001 real 0m3.656s user 0m0.016s sys 0m1.938s 'ls -l' 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 7 seconds llite.lustre-ffff9ba903dfd000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.batch_stats= snapshot_time: 1781664217.211839404 secs.nsecs start_time: 1781664104.148087046 secs.nsecs elapsed_time: 113.063752358 secs.nsecs subreqs per batch batches %% cum %% 1: 1 14 14 2: 0 0 14 4: 2 28 42 8: 1 14 57 16: 1 14 71 32: 0 0 71 64: 0 0 71 128: 0 0 71 256: 2 28 100 mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.batch_stats= snapshot_time: 1781664217.211896207 secs.nsecs start_time: 1781664104.148202909 secs.nsecs elapsed_time: 113.063693298 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 10 10 8: 3 30 40 16: 1 10 50 32: 2 20 70 64: 1 10 80 128: 0 0 80 256: 2 20 100 - open/close 4651 (time 1781664228.09 total 10.00 last 465.08) - open/close 9355 (time 1781664238.09 total 20.00 last 470.32) total: 10000 open/close in 21.31 seconds: 469.28 ops/second llite.lustre-ffff9ba903dfd000.statahead_batch_max=0 llite.lustre-ffff9ba903dfd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.stats=clear mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m41.381s user 0m0.136s sys 0m34.756s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9ba903dfd000.statahead_batch_max=256 llite.lustre-ffff9ba903dfd000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.stats=clear mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m34.515s user 0m0.142s sys 0m23.925s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9ba903dfd000.batch_stats= snapshot_time: 1781664324.590620824 secs.nsecs start_time: 1781664283.497692728 secs.nsecs elapsed_time: 41.092928096 secs.nsecs subreqs per batch batches %% cum %% 1: 417 69 69 2: 90 15 84 4: 29 4 89 8: 4 0 90 16: 3 0 90 32: 21 3 94 64: 2 0 94 128: 0 0 94 256: 34 5 100 mdc.lustre-MDT0001-mdc-ffff9ba903dfd000.batch_stats= snapshot_time: 1781664324.590666442 secs.nsecs start_time: 1781664283.497739391 secs.nsecs elapsed_time: 41.092927051 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9ba903dfd000.statahead_batch_max=64 llite.lustre-ffff9ba903dfd000.statahead_max=128