Five Kissmetrics Reports You Can Use to Guide Your Product Development

If you're building a SaaS product, getting constant feedback from customers is likely an integral part of your business.

There are two types of feedback – qualitative and quantitative.

Qualitative feedback is the reason people do what they do. You'll get this through your customer development, user research, etc. This feedback answers questions like:

• What do customers love about the product?
• Why did customers spend extra time on a certain page?
• What and where are customer frustrations?

Quantitative feedback is the numbers – the metrics. The Kissmetrics reports that you will get provide data about how customers are using your product. This feedback answers questions like:

• How are customers using the product?
• How often do customers use the product?
• Which features of the product do customers use?

There are times when you need to rely on one (qualitative or quantitative) more than the other, but you should never abandon either.

Kissmetrics can be used to gather quantitative feedback. Through the various reports, you can get valuable data that will help you build a better product. And since the reports provide the identities of people in the data, you will be able to contact them to get qualitative feedback as well.

For those who want the TL;DR, I created a small "Takeaways" summary at the end of each section. Read that if you want the CliffsNotes.

Now let's learn how you can get valuable data that will help you build a better product.

1. Find Your Power Users

Your user engagement may be all over the place. Some users may use your product every day, some a couple of times a week, and others a couple of times a month.

The ones who use your product every day (or on a more frequent basis than other groups of users) are known as your power users. These are your must-have customers. They value your product, are unlikely to churn, and probably tell their friends about you.

You need to know who these people are so that you can talk to them and get qualitative feedback. You want to discover what they like and don’t like about your product, what problems they solve using your product, and what frustrations they are having, etc.

You can also use these people for feedback on new feature releases. If you have a new feature but aren't ready to release it to your entire customer base, you can keep it limited to your power users. Gather feedback from them and iterate based on what they say.

Using the Kissmetrics People Search, you can find people based on criteria you set. For example, at Kissmetrics, we have an event called "Created Report." This event triggers any time a user creates a report. If we want to find our power users, we look for the people who created a report at least 7 times in the last 30 days. We can add other data to our report, too, such as how many reports they created, how many times they logged in, etc.

Here's how our criteria would look in People Search:

Again, our criteria is:

• We want to find the people who created a Kissmetrics report at least 7 times in the last 30 days.
• We want to add additional data to our search – the number of times each person created a report in the last 30 days and the number of times they logged in.

Let's click Search and get our list:

The left side (the blurred out section) is a list of customer email addresses. We can see each customer who fits this criteria and send them an email to learn why they do the things they do and how they feel about the product. The two columns on the right tell us how many times they created a report and how many times they logged in during the last 30 days.

Takeaways:

• How often people use your product will vary. Some may use it every day while others will use it only a few times a year. The people who use it most often are your power users. You want to know who these people are so you can talk to them.
• The Kissmetrics People Search helps you find these people.
• Once you use People Search and know who your power users are, you can talk to them and ask what they like and don’t like about your product, what problems it solves for them, and what frustrations they are having. This can also help you establish product/market fit so you can go after more of these customers.

2. Identify the People Who Don't Upgrade

Continuing with our feedback theme, it's also important to gather qualitative feedback from users who do not upgrade to paying customers. This can uncover some important insights:

• You can learn if you're targeting the right people for your product.
• You can find out if users ran into any trouble with your product or if it was not the right fit for them.
• You can gather any other feedback necessary to learn why users chose not to pay for your product. Maybe they were oversold or maybe there were problems with onboarding.

We don't need to change reports to get this data. With the People Search, we'll find the people who signed up but were never billed. Signed Up and Billed are not automatically tracked in Kissmetrics. If you're a SaaS company, you'll need to know when people sign up and when they are billed. This will be automatically recorded once you create the events.

Here's how our criteria would look:

Again, we want to find the people who signed up for a trial but were not billed. We can add other data, too, such as which marketing campaign they came from (with UTM parameters), when they signed up, etc. For now, let’s click Search and get our list:

The whole process takes less than 10 seconds.

It's a good idea to make this a routine process in your product development. Talking to the people who decide not to pay for your service is just as important as talking to the ones who love your product.

Takeaways:

• There are people who will sign up for a free trial (assuming you offer one) of your product but not upgrade to paying for it.
• You'll want to talk to these people and find out why. Were they onboarded incorrectly? Was the product not the right fit? Were they oversold? Were they confused about something and couldn't get past it?
• This feedback can be invaluable when iterating on your product. It can help you prioritize what needs the most attention (if it is something within your product that people didn't like).
• The People Search will find users who signed up but were not billed.

3. See Which Features People Are Using

Knowing which features are the most "sticky" can help you prioritize your product development and feedback cycle.

The Kissmetrics Cohort Report will help you find out which features are the "stickiest."

A cohort is a group of people who share a common characteristic or experience within a defined period of time. For example, people who signed up for your product during the month of April may be put in the "April Signup Cohort."

With the Kissmetrics Cohort Report, you can track which features are used repeatedly. You'll see the people who used a feature and then track how many of them came back to use it again. And since Kissmetrics is a people-tracking platform, you’ll know you're getting accurate data. Every person who comes to your site is identified as a person, not a pageview.

To track feature retention, just create an event that triggers any time someone uses a feature, and then segment by which feature they used.

From there, create your Cohort Report that looks at people who used the feature and then used the feature again. Here's how the setup would look with the Kissmetrics Cohort Report:

We're tracking use over time, which allows us to see if feature engagement increases or decreases over a period of months.

Here's the data visualization:

On the left, we see the feature type and how many people used that feature type during the selected date range. On the right, we see the percentage of those people who came back and used the feature again. The darker the shade of blue, the better.

Here are a few observations about this data:

• Feature G has a lot of engagement, but not a lot of people are using it. If we can acquire more of the type of people who use Feature G, chances are we’ll be acquiring loyal customers.
• A lot of people are using Feature A, and its engagement is on par with most of the other features.
• Over time, the majority of engagement on these features fades. To build a strong product, we'll need consistent engagement over time.
• Some of this data may come from trial users who used a feature but never returned. That can negatively affect the data. To make sure we see only current customers, we can use a filter (also called a Global Filter) so that we'll see usage data from paying customers only.

Takeaways:

• You want to build a lean product and remove any fat. Fat is features or areas of your product that no one uses. Think of it as Deals in Facebook. There wasn't a significant number of people using it, so they removed it.
• The Kissmetrics Cohort Report will help you know which features people are using consistently and which ones they are not.

4. See How Users Move Through a Product Flow

Many features in a SaaS product contain at least two steps. A user has to load the feature and then use, or complete, the feature. Here are a couple of examples:

1. When sending an email in Pardot or MailChimp, there are a few steps to go through before you can send (or schedule) the email.
2. Creating a report in Kissmetrics takes a few steps. You have to load the report, then set your criteria, and then run the report.

You get the idea. A user inevitably has to go through a few steps when using a feature.

The best way to track a user moving through a flow on your website is with the Kissmetrics Funnel Report. You'll see each step, from beginning to end, and you’ll see the percentage of users who move through the step.

Here's an example:

This Funnel Report has three steps:

1. Someone initially loaded the feature.
2. The person used the feature in some way.
3. The person completed a final action within the feature interface.

We see that during the month of September, a total of 2,500 people loaded the feature interface. But only about 50% of the people went through the entire product flow. Considering that this funnel has only 3 steps, there is much room for improvement. The majority of people who loaded the feature should complete their usage of it.

There are a few options we have that will improve our feature utilization:

• Get User Feedback – There's no shortage of ways to collect user feedback. Run through the product flow with a prospect and customer and listen to everything they say.
• A/B Test – This one is a little trickier, but with a little work you can pull it off. You can create an A/B test in the feature and see if your variant brings more people through the entire flow. You don't have to leave Kissmetrics to get this data – just use the A/B Test Report. It makes it really easy. You can test any part of your funnel with this report.
• Get the Right People to Use Your Product – Problems of low feature utilization and poor engagement can run deep. Perhaps you don't have product/market fit, users aren't being onboarded correctly, or they were oversold. Whatever the case, you need to make sure your target audience is using your product.
• Dig Deeper into the Data – While it is useful to see the percentage of users who move along this flow, we can also segment the data and see how different groups of users move along the flow. This is done with properties in Kissmetrics. You can create any property you want. Let's say we have a property for plan size. This segments people by the plan size they are on. We have four plans – small, medium, large, and enterprise. Let's segment our funnel by plan size:

Most of the segments, except for one, have an average flow. Around 90% of customers enter step 2, and about half of them complete using the feature. Enterprise customers are the only exception. Many of them move to the second step, but then there's a big drop-off. We'll need to investigate why.

Of course, we aren’t limited to segmenting by plan size. We can look at signup dates and see if the drop-off comes only from new customers. Or, we can look at browsers and see if the people dropping off are using a certain browser. Or, we can segment by free trial users and current customers. Maybe it's only free trial users who are struggling. As long as you're tracking a property in Kissmetrics, you can use it in your reports.

With the Funnel Report, you'll also be able to see exactly who is getting stuck in any step of a funnel. Maybe you want to see the people who get stuck after the first step. No problem, with the Funnel Report you can easily view these people. They should be a gold mine of great feedback.

Takeaways:

• Each feature in your product likely has a few steps users need to go through before they can "complete" using the feature.
• The Kissmetrics Funnel Report can be used to track how users move through a flow.
• There are a number of things you can do to improve feature utilization, and Kissmetrics will help you find the right people you need to talk to.
• You can segment data to get additional insights.
• Since Kissmetrics identifies each person, you can select any step in a funnel and see exactly who got stuck. Get the list of people within 30 seconds.

5. Find the People Who Engaged with a New Iteration

As a product manager, you're always iterating on features, adding something here, removing something there, and improving something somewhere. Are you making all these iterations in the dark, or are you tracking your customers to see if they are picking up and interacting with the iterations?

It's best to track who is using your product features and the iterations. Then you'll know who can give you valuable feedback. You can also find out how many times they used the iteration and the total number of customers using the iteration.

Let me give you a real world example.

At Kissmetrics, we recently iterated on Engage. Previously, customers had two colors to choose from – light and dark. We just added the ability to use custom colors. With this feature, customers can create any color combination they want with an Engagement.

We want to know who has used this new feature and how many times they've used it.

I'm sure you have a similar need. You don't want to release iterations and never know if anyone is using them. You want to know who, if anyone, is using them.

People Search is here to help.

You know the drill by now. People Search allows you to find users who have done certain events on your website. You set the criteria, and then you get a list of the people who met that criteria. If no one did, your search results will tell you that. Pretty simple.

To make sure this works, you'll need to create an event that triggers when a person uses the new iteration. For this example, an event triggers when someone uses custom colors. When we use People Search, we can see who is using the new iteration and how often. We also use the Funnel Report to see how many of the people who use custom colors actually complete the flow and launch their Engagement.

Now let’s look at another example. Let's say that we're a product manager at Twitter and we just released the "Favorite" option last week to a few thousand users. We want to see who is adopting it and how many times they are using it.

In the report setup, we’ll want to find the people who “favorited” a tweet (triggered the event). Here's the search criteria we would use to get the data:

We're looking at the people who have done the event "Favorited Tweet." We want to see all the people who have done this over the last 7 days. Also, we added more data to our search results to see how many times they favorited a tweet.

We'll click Search and get our data:

We get a list of people who favorited a tweet, as well as how many times they favorited a tweet.

This is great data, but to make it more useful, we need some context. We need to know how many times people have used Twitter. If they've logged in 50 times over the past 7 days but favorited only one tweet, we know it's getting low adoption. So we need to add additional data to this search.

Side note: Don't take this example too literally. I'm using Twitter because (a) everyone knows what it is and how it works and (b) there is a lot of crossover between it and other products. You may know people are using some new iteration in a larger feature of yours. And you may know how often they've used it. But you also need to know how often they use your overall product. That's what we're getting into here.

We'll add more data to our search results and see how many times each person logged in.

We can now get a glimpse of just how often users are favoriting tweets. We can also export this data to a CSV file to get the complete picture of all users.

Takeaways:

• Product managers iterate on features. Don’t be left in the dark. Track people to find out if anyone is using the new iterations.
• With People Search, you'll be able to find who used a new iteration and how many times they used it.

Let Data Help Guide Your Product Development

Product development isn't easy. As a data analytics platform, Kissmetrics will help you build a more effective product.

And if you're a marketer, Kissmetrics will be one of your favorite tools. We've built Kissmetrics for marketers. Head on over to the Kissmetrics Academy section to see exactly how Kissmetrics can help you optimize your marketing.

Interested in learning more about Kissmetrics? Then click on the button below to request a personal demo.

About the Author: Zach Bulygo (Twitter) is the Blog Manager for Kissmetrics.