Hey everyone! Carlos again. If you’re just starting to learn about data, or you’d like a refresher, this is the perfect place to start. Knowing the difference between qualitative is quantitative data is a great first step in your journey to becoming more data-driven.
Data is not the easiest field to master. Far from it, data science is as vast as it is complex. Luckily, as a Product Manager, no one expects you to be a full-blown data scientist. But the
best Product Managers are data-driven, and there’s no doubt that the more you understand data, the easier your job in PM will be.
If you’re just starting on your data journey, there are a few key things that you absolutely have to understand. The first is the difference between quantitative and qualitative data. Because at the end of the day, all data are either qualitative or quantitative. By understanding the difference between them, and knowing some basic research methods, you’re well on your way to becoming a data-driven Product Manager.
Check out: Stand Out as a PM Using Data
Why Data Matters
You might be asking yourself ‘Do I really need to know that much about data? That’s what the Data Scientist is for.’ The answer to that is, yes, you really do need to know about data!
There are many ways in which data help Product Managers on a daily basis. Just to give you a refresher, we’re going to go over the most important ones.
Data help Product Managers gain alignment. We tend to see data as ‘fact’ or something which can be used as objective truth when teams are trying to make decisions. While it’s rarely that simple, it’s true that data can help to provide context. When strong opinions are flying around the room, it’s useful to have some data sets to refer to, to aid with prioritization decisions.
Data add weight to your opinions. ‘Without data, you’re just another person with an opinion,’ as W. Edwards Deming tells us. In Product Management, you have to be able to influence without authority. Data can help you do that by providing evidence for your opinions.
Data help you make the right choices. It’s not just about looking like you know your stuff, it’s about actually knowing your stuff. Using data properly can ensure that you’re building the right things, for the right people, in the right way.
Data help you discover new opportunities. If used in the right way, data can challenge your perceptions and completely change your perspective, opening doors to new opportunities.
Data ensure you’re customer-focused. When you rely on the data, you’re getting to the truth about your customers, and letting the numbers tell you what customers really need, want, and value, as well as what will keep them around.
Data help you find your product-market fit. Good data gathering begins long before the start of the development process! When you perfect data analysis on your target marketing, and perform continuous data gathering on the market throughout development, you ensure that what you’re building has a place in the market. Find out why product-market fit is so vital for product success.
So now you’re fully aware of the myriad advantages that data can bring, and you’re ready to begin your journey to becoming data-driven.
The first thing you need to understand is that all data fit into one of two categories…qualitative or quantitative.
What Are Qualitative Data?
Qualitative data, also sometimes known as soft data, involve information that can’t be easily boiled down to numerical data.
Typical qualitative data sources might include:
App/Play store reviews
Press and PR
Beta tester feedback
Survey and interview questions
Customer service feedback
Qualitative data are useful because they capture things that the numbers can’t tell you, and fill in some vital gaps.
They’re usually loosely structured, and you’re unlikely to find a spreadsheet comprised fully of qualitative data. Typical qualitative data visualizations include word clouds and mood boards.
This type of data can also help you to form customer problem statements, as you can take words straight from the horse’s mouth. If you notice that various customers use the word ‘complicated’ to describe a process they’re having problems with, use that to form your problem statement. This keeps your problem statements directly related to what’s relevant to the customer, and not to you.
Analyzing qualitative data sounds difficult, but it adds a lot of value to your data sets, and helps you to remember that you’re building products for people and not for robots.
Imagine that you’re trying to invent the perfect cup of coffee, and so you set up a focus group in your local coffee shop. The qualitative feedback you gather might look something like this:
Common Qualitative Research Methods
Let’s go over some of the most common qualitative methods for data collection.
Interviews allow you to get up close and personal with your target (or existing) users. They add the human element to your user research. You can take the time to find out more about user problems, specific challenges they face, and their reaction to your products.
Focus groups give you an opportunity to speak to your target or existing users in a more time-effective way, as one-to-one interviews are more time-intensive. You can place a beta version of your product right into your users’ hands, and see how they interact with it. It can also be a useful exercise to see how people talk about the product when with other users, as when talking to the development team they may be telling you what they think you want to hear.
You can also plan a particular activity, like card-sorting, or run an A/B test, giving one focus group version A of the product and another group version B to measure if the changes between the two make a difference in user behavior.
Surveys can provide you with a decent mix of qualitative and quantitative data, if structured accordingly. By asking open-ended questions, and allowing users to elaborate, you can gather some key qualitative data. Don’t just ask, ‘do you like the product?’ Instead ask, ‘how does the product make you feel?’ ‘What would you improve about X or Y?’
Surveys are also popular with product teams, as you can gather a large amount of data without having to put in too much effort. Face-to-face interviews and focus groups take up quite a significant amount of time and resources, whereas a survey can be sent out to as many customers as you need, and collects data with little effort on your part.
What Are Quantitative Data?
Quantitative data, also known as hard data, are comprised of more traditional types of data, usually expressed in numeric values.
Quantitative data are the answer to closed-ended questions, and might involve data collection methods such as:
A/B test results
Most product teams rely on quantitative data because they’re seen as more reliable, and closer to the objective truth. While many different interpretations can be made of the same qualitative data set, you’ll generally find gaining alignment on quantitative data easier.
For example, if 99% of users who tried a new feature used it properly and achieved their goal, you can safely say the feature is easy to use, or is at least well designed. On the other hand, your whole team could read the same 3-star review on the App store and come away with a slightly different interpretation.
Let’s go back to our experiment in the coffee shop. The quantitative data which you collect from the same focus group might look something like this:
The Different Types of Quantitative Data
Quantitative data can be broken down into two different types:
Discrete data. Anything that you can count rather than measure, and is finite. For example, the number of daily active users you have, the number of people who check out vs the number of people who abandon cart, etc. With discrete data, the numbers normally can’t be broken down into smaller parts. (You can’t really have 0.25 of a user!)
Continuous data. These are data that constantly change, fluctuating between two presumed points. Examples of continuous data could be the time it takes to download something from your website, your age, or the amount of oxygen in your blood. These data can be broken down into smaller parts, which is critical in fields like medicine and engineering where accuracy is absolutely critical.
How to Use Both, And Why You Need To
Let’s finish off with our coffee shop experiment. When you bring the soft and hard data together, then you can start making the right assumptions to get the right insights:
Both quantitative and qualitative research are vital for fully fleshed-out ideation. You need to build something that both 1) solves a problem and 2) feels good to use.
Smart data-driven Product Managers don’t just use these data sets in isolation, they use them to complement each other. Use quantitative data to verify or challenge what users are telling you in reviews and interviews.
Many modern BI and Data Visualization platforms give you new ways to store, collect, and analyze your qualitative data. Qualitative data will also be incredibly helpful to teams such as marketing and PR; and your UX Designers will be able to use it to craft the perfect user personas.
Qualitative data are often seen as less important, but the truth is that they give you the voice of your customers. Using them alongside your quantitative data gives you a broader perspective and a more complete understanding of your customers, the market, and your product.
Congrats! You’ve just taken the first step towards becoming a data-driven PM! Come back to this guide any time to refresh your understanding.
Got anything interesting to add? Share your thoughts in the comment section below.
Don’t forget to check out some of the previous issues!