== sanity test 123ad: Verify batching statahead works correctly ========================================================== 11:49:15 (1788364155) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff93b06b24e800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93b06b24e800.batch_stats=0 llite.lustre-ffff93b06b24e800.statahead_max=32 llite.lustre-ffff93b06b24e800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff93b06b24e800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93b06b24e800.batch_stats=0 striped dir -i1 -c2 -H crush /mnt/lustre/d123ad.sanity total: 100 open/close in 1.16 seconds: 85.90 ops/second llite.lustre-ffff93b06b24e800.statahead_max=0 101 real 0m0.868s user 0m0.005s sys 0m0.286s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff93b06b24e800.statahead_max=32 llite.lustre-ffff93b06b24e800.statahead_max=32 101 real 0m0.431s user 0m0.004s sys 0m0.241s 'ls -l' 100 files with statahead: 1 sec statahead total: 86 statahead wrong: 2 agl total: 81 list_total: 29 fname_total: 43 hit_total: 13773 miss_total: 1468 swrong=2 ewrong=2 total: 900 open/close in 4.33 seconds: 207.99 ops/second llite.lustre-ffff93b06b24e800.statahead_max=0 1001 real 0m10.022s user 0m0.028s sys 0m2.934s 'ls -l' 1000 files without statahead: 10 sec llite.lustre-ffff93b06b24e800.statahead_max=32 llite.lustre-ffff93b06b24e800.statahead_max=32 1001 real 0m4.116s user 0m0.019s sys 0m1.770s 'ls -l' 1000 files with statahead: 4 sec statahead total: 87 statahead wrong: 2 agl total: 82 list_total: 30 fname_total: 43 hit_total: 14772 miss_total: 1469 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 8 seconds llite.lustre-ffff93b06b24e800.statahead_stats= statahead total: 87 statahead wrong: 2 agl total: 82 list_total: 30 fname_total: 43 hit_total: 14772 miss_total: 1469 mdc.lustre-MDT0000-mdc-ffff93b06b24e800.batch_stats= snapshot_time: 1788364193.177974232 secs.nsecs start_time: 1788364157.853658897 secs.nsecs elapsed_time: 35.324315335 secs.nsecs subreqs per batch batches %% cum %% 1: 5 12 12 2: 1 2 15 4: 1 2 17 8: 2 5 23 16: 23 58 82 32: 7 17 100 mdc.lustre-MDT0001-mdc-ffff93b06b24e800.batch_stats= snapshot_time: 1788364193.178021464 secs.nsecs start_time: 1788364157.853743273 secs.nsecs elapsed_time: 35.324278191 secs.nsecs subreqs per batch batches %% cum %% 1: 2 5 5 2: 0 0 5 4: 1 2 7 8: 4 10 17 16: 10 25 42 32: 23 57 100 - open/close 2743 (time 1788364204.18 total 10.00 last 274.21) - open/close 5150 (time 1788364214.18 total 20.01 last 240.65) - open/close 7625 (time 1788364224.19 total 30.01 last 247.46) total: 10000 open/close in 39.84 seconds: 250.99 ops/second llite.lustre-ffff93b06b24e800.statahead_batch_max=0 llite.lustre-ffff93b06b24e800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93b06b24e800.stats=clear mdc.lustre-MDT0001-mdc-ffff93b06b24e800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m33.327s user 0m0.230s sys 0m30.217s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93b06b24e800.statahead_batch_max=32 llite.lustre-ffff93b06b24e800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93b06b24e800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff93b06b24e800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93b06b24e800.stats=clear mdc.lustre-MDT0001-mdc-ffff93b06b24e800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m38.492s user 0m0.195s sys 0m21.373s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff93b06b24e800.batch_stats= snapshot_time: 1788364315.799875695 secs.nsecs start_time: 1788364270.974321703 secs.nsecs elapsed_time: 44.825553992 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 2 0 0 32: 312 99 100 mdc.lustre-MDT0001-mdc-ffff93b06b24e800.batch_stats= snapshot_time: 1788364315.799918593 secs.nsecs start_time: 1788364270.974413130 secs.nsecs elapsed_time: 44.825505463 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff93b06b24e800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93b06b24e800.batch_stats=0 llite.lustre-ffff93b06b24e800.statahead_max=2048 llite.lustre-ffff93b06b24e800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff93b06b24e800.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff93b06b24e800.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.10 seconds: 90.52 ops/second llite.lustre-ffff93b06b24e800.statahead_max=0 101 real 0m0.819s user 0m0.002s sys 0m0.309s 'ls -l' 100 files without statahead: 1 sec llite.lustre-ffff93b06b24e800.statahead_max=2048 llite.lustre-ffff93b06b24e800.statahead_max=2048 101 real 0m0.364s user 0m0.000s sys 0m0.229s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10098 miss_total: 2 swrong=0 ewrong=0 total: 900 open/close in 2.10 seconds: 429.29 ops/second llite.lustre-ffff93b06b24e800.statahead_max=0 1001 real 0m7.306s user 0m0.033s sys 0m2.608s 'ls -l' 1000 files without statahead: 7 sec llite.lustre-ffff93b06b24e800.statahead_max=2048 llite.lustre-ffff93b06b24e800.statahead_max=2048 1001 real 0m3.897s user 0m0.013s sys 0m2.202s 'ls -l' 1000 files with statahead: 4 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 6 seconds llite.lustre-ffff93b06b24e800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11097 miss_total: 3 mdc.lustre-MDT0000-mdc-ffff93b06b24e800.batch_stats= snapshot_time: 1788364438.609067161 secs.nsecs start_time: 1788364316.702092011 secs.nsecs elapsed_time: 121.906975150 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 1 12 12 4: 2 25 37 8: 1 12 50 16: 1 12 62 32: 0 0 62 64: 1 12 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff93b06b24e800.batch_stats= snapshot_time: 1788364438.609184660 secs.nsecs start_time: 1788364316.702247549 secs.nsecs elapsed_time: 121.906937111 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 1 12 12 8: 4 50 62 16: 0 0 62 32: 1 12 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 - open/close 3779 (time 1788364449.60 total 10.01 last 377.62) - open/close 6717 (time 1788364459.60 total 20.01 last 293.78) total: 10000 open/close in 28.28 seconds: 353.56 ops/second llite.lustre-ffff93b06b24e800.statahead_batch_max=0 llite.lustre-ffff93b06b24e800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93b06b24e800.stats=clear mdc.lustre-MDT0001-mdc-ffff93b06b24e800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m31.733s user 0m0.181s sys 0m27.446s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff93b06b24e800.statahead_batch_max=256 llite.lustre-ffff93b06b24e800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff93b06b24e800.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff93b06b24e800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff93b06b24e800.stats=clear mdc.lustre-MDT0001-mdc-ffff93b06b24e800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m27.645s user 0m0.170s sys 0m20.575s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff93b06b24e800.batch_stats= snapshot_time: 1788364537.156293429 secs.nsecs start_time: 1788364503.067260884 secs.nsecs elapsed_time: 34.089032545 secs.nsecs subreqs per batch batches %% cum %% 1: 319 56 56 2: 103 18 74 4: 68 12 86 8: 13 2 89 16: 4 0 89 32: 21 3 93 64: 4 0 94 128: 0 0 94 256: 33 5 100 mdc.lustre-MDT0001-mdc-ffff93b06b24e800.batch_stats= snapshot_time: 1788364537.156349472 secs.nsecs start_time: 1788364503.067336156 secs.nsecs elapsed_time: 34.089013316 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff93b06b24e800.statahead_batch_max=64 llite.lustre-ffff93b06b24e800.statahead_max=128