Striving for accuracy in forecasting analytics

Mark  Ryan

March 13, 2011

Related Topics

Forecasting results is an important part of a new initiative. The forecasting often starts with a simple comparison of a current benchmark with a projected improvement.

1. Benchmark:
Current value of a conversion event * the current frequency of a conversion event

2. Estimate an improvement
Current value of a conversion event * the future frequency of a conversion event
Let's say our goal is to increase lead generation and we want to increase lead generation by optimizing our onsite/internal search results. A forecast might look like this:

Benchmark

Visitor Searchers Lead
Conversion
Converted Customer
Conversion
Sale
Value
1 YR
Total Value
22,000,000 2,200,000 2.25% 49,500 1.50% $7,000 $5,197,500

Estimate for improvement

Visitor Searchers Lead
Conversion
Converted Customer
Conversion
Sale
Value
1 YR
Total Value
22,000,000 2,200,000 2.50% 55,000 1.50% $7,000 $5,775,000

In this estimate, we are showing that if we can increase the visitor to lead conversion ratio, by optimizing the search tool, by .25%, we will derive a value of approximately $577,500 after 1 year.

The big question
How accurate of a prediction is it to increase this ratio by .25%? This, of course, is an isolated example. But most forecasts have to show some sort of improvement and it begs the question of 'is that improvement feasible or likely'? Often the analyst doing the forecasting will pick a conservative level of improvement to show the value of a particular initiative.

Extractable uses improvement data from previous/similar engagements, public case studies, and industry research to forecast improvements. But every site is unique, every customer base is fairly unique, products are unique to the company, sales cycles are unique to the organization. It's healthy to ask, 'just because this improvement worked elsewhere, does that mean it will work with us'?

Going back to the example of search and lead generation, many of the major search providers have case studies showing 20% increases in lead generation once a particular search solution was implemented. Obviously, not every customer of theirs has or can have the same boost.

In some cases we will show 3 forecasts. The first forecast will show the value of an improvement based on what we have seen on similar engagements. The 2nd will show the forecast based on what we have seen in industry published research. The 3rd will show the forecast based on case studies published from a specific technology vendor. From there, the teams will collaborate and make a 'best guess' on which level of improvement seems most likely. This technique has worked well for us over the last several years and has allowed us to be more accurate with our forecasts.