== sanity test 123ad: Verify batching statahead works correctly ========================================================== 09:53:35 (1787147615) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff928dc6d65000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff928dc6d65000.batch_stats=0 llite.lustre-ffff928dc6d65000.statahead_max=32 llite.lustre-ffff928dc6d65000.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff928dc6d65000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff928dc6d65000.batch_stats=0 striped dir -i1 -c2 -H crush2 /mnt/lustre/d123ad.sanity total: 100 open/close in 2.32 seconds: 43.12 ops/second llite.lustre-ffff928dc6d65000.statahead_max=0 101 real 0m2.068s user 0m0.001s sys 0m0.302s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff928dc6d65000.statahead_max=32 llite.lustre-ffff928dc6d65000.statahead_max=32 101 real 0m0.936s user 0m0.005s sys 0m0.163s 'ls -l' 100 files with statahead: 1 sec statahead total: 118 statahead wrong: 2 agl total: 111 list_total: 30 fname_total: 88 hit_total: 4130 miss_total: 1395 swrong=2 ewrong=2 total: 900 open/close in 8.83 seconds: 101.96 ops/second llite.lustre-ffff928dc6d65000.statahead_max=0 1001 real 0m21.729s user 0m0.032s sys 0m2.924s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff928dc6d65000.statahead_max=32 llite.lustre-ffff928dc6d65000.statahead_max=32 1001 real 0m9.633s user 0m0.028s sys 0m1.388s 'ls -l' 1000 files with statahead: 10 sec statahead total: 119 statahead wrong: 2 agl total: 112 list_total: 31 fname_total: 88 hit_total: 5127 miss_total: 1398 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 18 seconds llite.lustre-ffff928dc6d65000.statahead_stats= statahead total: 119 statahead wrong: 2 agl total: 112 list_total: 31 fname_total: 88 hit_total: 5127 miss_total: 1398 mdc.lustre-MDT0000-mdc-ffff928dc6d65000.batch_stats= snapshot_time: 1787147702.029956989 secs.nsecs start_time: 1787147619.745016098 secs.nsecs elapsed_time: 82.284940891 secs.nsecs subreqs per batch batches %% cum %% 1: 3 7 7 2: 1 2 10 4: 2 5 15 8: 2 5 21 16: 24 63 84 32: 6 15 100 mdc.lustre-MDT0001-mdc-ffff928dc6d65000.batch_stats= snapshot_time: 1787147702.030072195 secs.nsecs start_time: 1787147619.745318298 secs.nsecs elapsed_time: 82.284753897 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 4 10 12 16: 13 33 46 32: 21 53 100 - open/close 865 (time 1787147714.64 total 10.01 last 86.43) - open/close 1784 (time 1787147724.65 total 20.02 last 91.81) - open/close 2749 (time 1787147734.66 total 30.03 last 96.38) - open/close 3559 (time 1787147744.67 total 40.04 last 80.96) - open/close 4442 (time 1787147754.67 total 50.04 last 88.26) - open/close 5204 (time 1787147764.69 total 60.06 last 76.02) - open/close 6028 (time 1787147774.70 total 70.07 last 82.36) - open/close 6750 (time 1787147784.72 total 80.09 last 72.06) - open/close 7612 (time 1787147794.73 total 90.10 last 86.10) - open/close 8436 (time 1787147804.74 total 100.11 last 82.31) - open/close 9380 (time 1787147814.75 total 110.11 last 94.36) total: 10000 open/close in 116.45 seconds: 85.87 ops/second llite.lustre-ffff928dc6d65000.statahead_batch_max=0 llite.lustre-ffff928dc6d65000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff928dc6d65000.stats=clear mdc.lustre-MDT0001-mdc-ffff928dc6d65000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m13.617s user 0m0.324s sys 0m14.787s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff928dc6d65000.statahead_batch_max=32 llite.lustre-ffff928dc6d65000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff928dc6d65000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff928dc6d65000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff928dc6d65000.stats=clear mdc.lustre-MDT0001-mdc-ffff928dc6d65000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m41.183s user 0m0.315s sys 0m13.420s batched statahead RPC= hit count=9981 mdc.lustre-MDT0000-mdc-ffff928dc6d65000.batch_stats= snapshot_time: 1787148016.269407529 secs.nsecs start_time: 1787147901.960808168 secs.nsecs elapsed_time: 114.308599361 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 mdc.lustre-MDT0001-mdc-ffff928dc6d65000.batch_stats= snapshot_time: 1787148016.269474921 secs.nsecs start_time: 1787147901.960899148 secs.nsecs elapsed_time: 114.308575773 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff928dc6d65000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff928dc6d65000.batch_stats=0 llite.lustre-ffff928dc6d65000.statahead_max=2048 llite.lustre-ffff928dc6d65000.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff928dc6d65000.batch_stats=0 mdc.lustre-MDT0001-mdc-ffff928dc6d65000.batch_stats=0 striped dir -i1 -c2 -H fnv_1a_64 /mnt/lustre/d123ad.sanity total: 100 open/close in 1.96 seconds: 51.05 ops/second llite.lustre-ffff928dc6d65000.statahead_max=0 101 real 0m1.694s user 0m0.004s sys 0m0.311s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff928dc6d65000.statahead_max=2048 llite.lustre-ffff928dc6d65000.statahead_max=2048 101 real 0m1.036s user 0m0.004s sys 0m0.149s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10080 miss_total: 20 swrong=0 ewrong=0 - open/close 834 (time 1787148308.40 total 10.00 last 83.38) total: 900 open/close in 10.83 seconds: 83.09 ops/second llite.lustre-ffff928dc6d65000.statahead_max=0 1001 real 0m22.056s user 0m0.038s sys 0m3.001s 'ls -l' 1000 files without statahead: 22 sec llite.lustre-ffff928dc6d65000.statahead_max=2048 llite.lustre-ffff928dc6d65000.statahead_max=2048 1001 real 0m7.466s user 0m0.033s sys 0m1.456s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 19 seconds llite.lustre-ffff928dc6d65000.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11079 miss_total: 21 mdc.lustre-MDT0000-mdc-ffff928dc6d65000.batch_stats= snapshot_time: 1787148371.367881093 secs.nsecs start_time: 1787148018.979726155 secs.nsecs elapsed_time: 352.388154938 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 25 25 8: 2 25 50 16: 1 12 62 32: 1 12 75 64: 0 0 75 128: 0 0 75 256: 2 25 100 mdc.lustre-MDT0001-mdc-ffff928dc6d65000.batch_stats= snapshot_time: 1787148371.367951112 secs.nsecs start_time: 1787148018.979875282 secs.nsecs elapsed_time: 352.388075830 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 2 25 25 8: 3 37 62 16: 0 0 62 32: 0 0 62 64: 1 12 75 128: 0 0 75 256: 2 25 100 - open/close 830 (time 1787148383.73 total 10.01 last 82.95) - open/close 1794 (time 1787148393.74 total 20.01 last 96.33) - open/close 2741 (time 1787148403.75 total 30.02 last 94.61) - open/close 3624 (time 1787148413.76 total 40.03 last 88.26) - open/close 4566 (time 1787148423.76 total 50.03 last 94.16) - open/close 5415 (time 1787148433.76 total 60.03 last 84.88) - open/close 6141 (time 1787148443.78 total 70.06 last 72.45) - open/close 6949 (time 1787148453.80 total 80.07 last 80.67) - open/close 7764 (time 1787148463.80 total 90.07 last 81.48) - open/close 8722 (time 1787148473.81 total 100.08 last 95.72) - open/close 9569 (time 1787148483.82 total 110.09 last 84.61) total: 10000 open/close in 115.13 seconds: 86.86 ops/second llite.lustre-ffff928dc6d65000.statahead_batch_max=0 llite.lustre-ffff928dc6d65000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff928dc6d65000.stats=clear mdc.lustre-MDT0001-mdc-ffff928dc6d65000.stats=clear ls -l files in d123ad.sanity: 10001 real 1m0.553s user 0m0.334s sys 0m14.416s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff928dc6d65000.statahead_batch_max=256 llite.lustre-ffff928dc6d65000.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff928dc6d65000.batch_stats=clear mdc.lustre-MDT0001-mdc-ffff928dc6d65000.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff928dc6d65000.stats=clear mdc.lustre-MDT0001-mdc-ffff928dc6d65000.stats=clear ls -l files in d123ad.sanity: 10001 real 0m49.195s user 0m0.281s sys 0m13.427s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff928dc6d65000.batch_stats= snapshot_time: 1787148615.496958329 secs.nsecs start_time: 1787148558.011730507 secs.nsecs elapsed_time: 57.485227822 secs.nsecs subreqs per batch batches %% cum %% 1: 115 42 42 2: 46 16 59 4: 30 11 70 8: 12 4 74 16: 6 2 76 32: 21 7 84 64: 5 1 86 128: 3 1 87 256: 34 12 100 mdc.lustre-MDT0001-mdc-ffff928dc6d65000.batch_stats= snapshot_time: 1787148615.497053829 secs.nsecs start_time: 1787148558.011878299 secs.nsecs elapsed_time: 57.485175530 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff928dc6d65000.statahead_batch_max=64 llite.lustre-ffff928dc6d65000.statahead_max=128