Numbers Aren’t Enough: How Qualitative Data Enhances Customer Insights

Meg Davis  

May 07, 2012

Related Topics

There are various ways companies learn about their customers. They fall generally into two kind of activities: quantitative and qualitative activities. Quantitative activities focus on measurable behaviors and include:

  • Web-site analytics
  • Sales numbers
  • CRM reporting
  • Surveys

Qualitative activities focus on understanding why these behaviors happen and include:

  • User interviews
  • Focus Groups
  • Eye-tracking
  • Ethnography
  • Usability Testing

According to an independent study Data Driven Design: Digital Experience Teams Are Focused On Website Metrics That Don't Demonstrate Business Value (April 2012), a commissioned study conducted by Forrester consulting on behalf of Extractable, "Many firms report that they don't use techniques like card sorting (46%), ethnography (44%), website session recording and playback tools (44%), eye tracking studies (44%), and usability lab testing (30%) (see Figure 8). Instead, they focus on techniques like search engine search term analysis (75%), behavioral web analytics (72%), and online survey and feedback tools (67%)."

At Extractable, we believe that the best customer insights come from the intersection of the quantitative data and qualitative data. Why is that?

At a high level, these are the differences in strengths between quantitative data and qualitative data:

Quantitative Data Strengths

Qualitative Data Strengths

Identifies the current state of what is happening and the current behavior of the customer

Identifies why it's happening and gives insight into the train of thought of the customer

Focused on present state and present behavior

Focused on what could be - the hypothetical

Examines the experience from the standpoint of the current structure

Examines the experience from the underlying needs of the customer and points out what you don't know you don't know

Concentrated on short-term optimization of the experience

Concentrated on long-term, fundamental changes of the experience



Analyzes very large sets of customers

Analyzes a small representative size of customers

How can the strengths of these different methods reinforce each other and yield better results?

  • Quantitative identifies areas of opportunity for qualitative. With a large sample size, quantitative can detect anomalies and patterns in customer behaviors. With this knowledge, qualitative analysis can focus on identifying why this behavior is happening at the core.  For example, in a recent project, analysis of web analytics showed that customers were visiting two pages back and forth in succession on the website. The two webpages were two different product pages. The short-term solution was to combine the information on these two pages. However, the question remained: what was driving people to want to see this information together? This was a question that we took into qualitative methods.
  • Qualitative explains quantitative. Qualitative data focuses on why the customer is behaving in a certain way. Once fundamental needs are discovered through qualitative methods, longer-term solutions can be proposed and tested. In our example, the long-term solution that qualitative research set out to find was why people were connecting these two product pages and what information in particular they were associating. Through qualitative usability testing observing prospective customers using the website, we found that fundamentally people did not understand the differences between the product offerings. A better explanation of the product offerings was needed at a higher level of the website, like the homepage.
  • Quantitative validates qualitative. Because qualitative is focused on a small representative sample set, quantitative can give the number results to quantify the success of a change. Quantitative data can help companies understand how changes have affected other customer behavior. In our example, after we added a better comparison and overview explanation of the product offerings of the company on the homepage, we saw less switching between pages and a more focused path that allowed the engaged visitor to get one level deeper into the site on average.
  • Quantitative allows for maintenance and optimization. Quantitative data can continue to monitor and optimize customer behavior in real-time to understand how customer behavior is changing and to understand when qualitative testing would be useful. In our example, we continued to monitor the company's analytics. Over time as the company's industry became better known, the need for explanation of products became less important. The new challenge was how to differentiate its products from other competitors.

There are many reasons companies don't pursue qualitative data. Chief among them are the large expense and the specialized skills needed. Because qualitative data requires sitting down with customers and listening to them as individuals, it can be time-consuming to moderate. The moderator must be sure not to lead customers to any conclusions but impartially guide the session. In addition, individuals' feedback must be analyzed for patterns in order to understand trends about how customers think.

However, there is no substitute for getting inside your customers' minds. Qualitative insights allow us to not only create a website that is usable and meets a customer's short-term goals, but a web experience that resonates with customers at a deeper emotional level and that meets their long-term goals for how it should feel to do fulfill the needs that your business offerings meet. Dig deeper and enhance your quantitative practices with qualitative methods.