Measuring the Success of a Mobile App

“How do you go about measuring the success of a mobile app?”

– John | Marketing Executive

Good question John.

In this world of big data it’s the marketer with the right metrics that gets the cheese.

Making a call on raw, unsegmented, broad, inaccurate or tampered data is a rookie mistake, and a fast way to kiss your job goodbye.

You can only hide within the marketing black-box for so long. But when your boss comes looking for an update on the latest mobile app campaign what do you intend to say:

“It’s going well…I think.”  ??

In this guide I want to show you:

1. how to run accurate numbers to drive a successful mobile marketing campaign, and
2. how to become a marketing leader within your organisation, in mobile AND all other mediums.

The Marketing Talent Gap in 2014

From the State of Digital Marketing Talent study OMI asked 747 advertising and Fortune 500 Marketing Executives which digital marketing skills they value the most. The results came in with a whopping 37% talent gap in ‘Analytics’.

Contrary to common belief, analytics is NOT about deciphering weird patterns from numbers, or crunching advanced algebraic formulas. It’s about:

1. Running live experiments to prove a series of hypotheses; then
2. Figuring out the right metrics for measuring the success of a campaign; and finally
3. Setting up systems that turns the raw data into easy to read reports.

This is the number one sought-after skill in marketing, world-wide.

If this is an area that you’re not 100% sure about or consider to be a weak-point then this guide is for you.

There are literally hundreds of reasons why businesses are investing into mobile applications. If you want the 10 most compelling reasons check out the link below.

See also: How Businesses Can take Advantage Of The Next $100B Market Opportunity

How to Measure the Success of a Mobile App

In this guide we’re going to explore the 5 key metrics you need to measure and quantify the success of a mobile application.

This will help you define the right sub-metrics for your campaign.

As I mentioned earlier, the goal is to set up a system that will assist you in making decisions. Or at least spits out a report that you can give to the person in charge of making these decisions.

Decisions about what improvements can be made to your campaign in order to (a) drive higher engagement, and (b) reach business objectives faster.

See also: The Startup Guide to App Development

The 5 Most Important Metrics for Measuring the Success of a Mobile App

We’re going to break down the 5 key metrics into sub-metrics so that you can tailor them for your own campaign.

Here are the key metrics you should be focusing on in measuring app success:

1. Acquisition Metrics
2. Engagement Metrics
3. Retention Metrics
4. Quality Metrics
5. Behaviour Metrics

By breaking down these metrics and segmenting the data you can set up systems that print meaningful reports day in / day out.

And if you’re running your own business, these numbers give you full transparency into the health of your businesses/application. Failure to collect the right data will drive even the biggest business into the ground.

Let’s dig down into these individual key metrics to find out how we can use them.

1. Acquisition Metrics

“How do I track users through an acquisition funnel?”

– Jane | Marketing Intern 

Acquisition can be broken down into two parts:

• Discovery
• Downloads

Downloads

It’s important to get large volume of quality downloads.

Finding traffic sources can be hard and generally requires some testing. For example, you may find that your particular audience doesn’t engage well on Facebook.

I personally recommend testing all viable channels multiple times during the app marketing campaign to find out:

1. Which channels bring in the most downloads; and
2. Which channels brings in the best quality users.

What is a Quality User?

A Quality User is defined as someone who completes one or more of the actions you want them to take.

See Engagement Metrics < Goals Completed below

That doesn’t mean that users who don’t take these actions are poor quality users.

If the user experience of your application is confusing then this will kill any chance of converting users into Quality Users and therefore throws your metrics right out of whack. Invest into good user experience design.

See also: 5 Quick Tips on App User Experience Design

Consider also that perhaps your User Persona / Customer Profile hasn’t been taken into consideration. Or maybe you don’t have one. Perhaps the experience you have created doesn’t match the mood of the customers you’re trying to attract.

See also: User Persona | Marketing to a Mobile Audience

Discovery

Understanding where your users are finding/discovering your application will help you start to define which channels are providing the best ROI. In other words, finding out where you should focus more marketing dollars, etc.

Deep tracking

Knowing where most users are coming from won’t give you accurate metrics on ROI.

If a user completes an action you want them to take (fill out a form, share to Facebook, etc.) you need to know:

1. Where that user came from (channel),
2. What steps he/she took within the app to get there,

With this understanding you can start driving users through that same experience and – by theory – achieve a higher conversion rate of desirable actions/goal completions.

It’s all well and good to get thousands of downloads per day, but if certain traffic channels aren’t converting into paying customers then you need to know which ones to cut off, which ones to tweak and which ones to focus more on.

If you are promoting through a number of channels (Youtube, Ads, Content marketing, Email, etc.) you can to set up tracking scripts that integrate with your mobile metrics / analytics tool.

Later, when the user has downloaded the app and they make a purchase, etc. you know exactly where they came from, what ad they saw and what the acquisition cost is of that user.

Remember, the basic principals of cost per acquisition, etc. still apply.

Why Acquisition Metrics are Important

User downloads is the lifeblood of your mobile marketing campaign.

Without an ample supply of users you cannot accurately segment, measure and track your mobile data. The same goes for websites, desktop applications, email campaigns, landing pages, blogs, social media, etc.

In other words, you can’t confidently draw trends in user behaviour from 100 users over 30 days.

The minimum number of monthly downloads for accurate data analysis is 3,000. This also depends on the size of the application. Bigger apps require more downloads.

2. Engagement Metrics

“How do I know if my users are engaged?”

– James | Startup Entrepreneur

Engagement can be broken down into 4 parts:

• Active users
• Session length
• Goals completed
• User loyalty

The idea is to get a c-section of user engagement data and then turn that into a ratio.

Over time (days, weeks, months…) you should be able to gauge whether your ‘user engagement’ is increasing…or decreasing.

First of all, don’t be afraid of negative numbers. It happens to the best of us. Even I hit up against decreases all the time.

The best way to look at a decrease is as a learning experience. A decrease is a sign that something has changed and your users are no longer engaging the way they use to.

It’s an opportunity to improve. Try new things. Get creative.

Find out what’s causing the decrease and fix it. Most likely a recent change to the app.

Active users

Every app marketer has their own definition of an active user. And it’s usually dependant on the general behaviour of their overall user base.

A generic definition of an active user that gets used a lot is someone who has opened the application 5 times or more. In other words, the user has had 5 sessions with your application.

This is where website marketing and mobile app marketing is different-

*Goal Completion – also known as ‘Events’ – is when the user completes an action you want them to take, i.e. Facebook share, etc.

As I mentioned, over time as you get to know your audience you’ll create your own definition for ‘Active User’.

Consider some of the following definitions:

• Has spent in excess of 30 minutes using the application
• Has used the application at least once per week for 3 months
• Uses the application daily
• Has demonstrated some other behaviour that deems him/her an engaged user

These things need to be programmed into the application. Ask your app development team to help you out.

Session length

This is an important metric.

As a marketer you know that the longer your users stay within your application the higher the chance is that they will complete a goal/event.

So then with that in mind, what can you do to keep your users engaged for a longer period of time? I have included some resources for further reading on how to engage your audience, below:

See: How to Gamify Your App

See: 6 Fast Tips to Mobile User Experience Design

Goals Completed

First of all, what is a goal? I have previously referred to goals as  events or desired actions taken by the user.

I prefer the term ‘Goal’ because it is exactly that – a goal you set for the user.

Now this can be nearly anything but you want it to be something of value. For example:

• Facebook share
• In-app purchase
• Click an ad
• Fill out a form
• Click through to a website
• Download content
• Request more information
• Etc.

Goals are wholly dependant on (a) the key objectives for the campaign, and (b) your over-arching strategy.

They SHOULD be directly related to the attainment of your key objectives. And your application design needs be set to lead users into completing these goals.

Goals are easy to track and can be programmed into the application using free app analytics tools.

I have provided a comparison list of the 10 Most Popular Mobile App Analytics Tools below. The tools are compared by features and pricing.

See: 10 Most Popular Mobile App Analytics Tools

User loyalty

User loyalty is a measure of the average time-span that someone remains an Active User of your application.

This is one of the biggest challenges for mobile apps, because 65% of people stop using apps 3 months after install.

The best way to build user loyalty is:

1. Integrate mobile gamification
2. Build a user-engagement strategy using push-notifications
3. Run split-testing campaigns within the application to increase engagement

See also: How to Build a Solid User Engagement Strategy

See also: 3 Quick Split-Tests to Double User Engagement

Why Engagement Metrics Are Important

If new users are the life blood of your mobile app then user engagement is the muscle that when fed with blood lets your business grow.

User engagement is what drives mobile enterprises. If customers aren’t engaged with your application then it’s not going to grow.

Period.

Sure you can run acquisition campaigns with conversion points within that turn users into $$$. Then you can show your boss the metrics that make the campaign seem like a success.

But the users will soon get tired of the app –> Eventually the market will be saturated with your ads –> Organic traffic will dry up and you’ll be scrounging for new acquisition channels…

Your application will be doomed.

You CANNOT build a mobile application campaign upon acquisition alone. You MUST have engagement.

User engagement is customer experience.

Do you know why I fly Virgin? Because I love the experience. Do you know why I shop at Woolworths instead of Aldi when it costs an average of 25% more? Because Aldi is gross and I’m superficial.

Experience is everything.

See also: 6 Hot Tips For Designing a Magnificent User Experience

3. Retention Metrics

“How do I track user retention and measure growth?”

– Antony | Strategy Director

This is a metric you want to keep a close eye on. The 2 sub-metrics you need to watch are:

1. Life-time value (LTV)
2. Retention rate

Let’s start with #1-

Life-time value (LTV)

If your retention rate is low, meaning you have a high churn rate, then hitting critical mass becomes an expensive ordeal.

A high retention rate increases the average LTV because you get a longer lasting user, but the cost per acquisition stays the same.

That’s not even taking into consideration the server costs or any other costs associated with user-volume.

When talking about 1’s and 2’s it’s not all so impressive but start dropping some zero’s in there and you’ll start seeing numbers that will either make you look really good…or really, really bad.

There are two ways to fix the LTV:

1. Increase revenue per user, the user generates more revenue for you during the time they are active in your app.
2. Increase long term active engagement, the user stays active on the application for longer.

Retention rate

The retention rate is a metric measured and displayed as a percentage (%).

In essence, if you have 100 users this month and in 30 days you managed to retain 80 of them then your retention rate is 80% (or 20% churn-rate).

Keep a close eye on your daily, weekly and monthly retention rates to gauge if there is any increase or decrease. These should be tracked with the same precaution as the User Loyalty metric.

Why Retention Metrics are Important

Engaging users is as important as captivating your target audience with a good TV advertisement. If you want people to connect with your brand then engagement is absolutely critical.

If your customers see your application as being something gimmicky that should be tossed aside then they may carry that connection over to your other products and/or services as well.

Plus the app marketplaces like the Apple App Store and Google Play are becoming more sophisticate and are starting to focus more on retention and engagement.

This means less fly-by-night apps on the App Charts and more quality apps.

SIDE NOTE: Market viability

A high retention rate is often an early sign of market viability. 

If people are adopting your application as a tool for daily life then you’re already half way there and it’s a good indicator that you’re heading in the right direction.

See also: 5 Tips for How to Validate Your Idea & Launch Quickly

4. Quality Metrics

“How do I know that my app isn’t crashing on the App Store?”

– Tina | Marketing Director

This metric refers to the quality of the code. Technical/user issues can be tracked & measured using 3 sub-metrics:

• Automatic crash-report
• Manual user feedback
• App store reviews

The first two options should be implemented into the application from the get-go. The latter needs to be monitored over time.

Automatic crash-report

An automatic crash-report should be programmed into the code of the application.

Just like anything else, you need to specifically ask your app developer to add this in because most of them won’t know to do this.

Automatic crash-reports means that every time the application crashes the issue will be logged and emailed to you and your app developer.

The app developer will then need to reverse engineer the issue to reproduce it. Once reproduced it can be fixed ready for the next update.

Manual user feedback

Automatic crash-reports only record issues that cause the application to crash. However, it doesn’t report issues in the user experience.

These can range from incorrect image sizing, feature malfunction, and pretty much anything under the user experience umbrella.

A quick fix for this is a user-feedback form that the users can fill in with any feedback.

A user-feedback form can automatically collect the following data:

• Time & date stamp.
• Location – could be an issue of connectivity as the user is outside of a satellite zone.
• Operating System (e.g. iOS, Android, Blackberry, etc.) and version (e.g. iOS 7.1).

Another piece of information you may need to collect is the actual device on which the issue occurred. Certain issues are unique to specific devices.

Offer a drop-down, swivel or text box in the form for the user to nominate what device they’re using.

App store reviews

This is the last place where you want to be receiving negative feedback. So take every measure you can to ensure all negative feedback is captured through the mediums above.

However, if you do receive negative feedback (and it’s inevitable) then make sure you act on it right away.

Why Quality Metrics are Important

Even when working with the most skilled app developers you’ll find your application will still have issues. It’s unpreventable.

Even the legendary programmers at Apple release iOS versions with major gaps and bugs. So expect bugs.

As soon as an issue occurs it’s best that it’s noted with a crash-report and sent directly to the developer, and to the marketer with a time-stamp.

This allows the marketing person to cross reference increases or decreases in engagement with temporary technical issues.

As technical issues affect the user experience you will need to keep a close eye on these reports and make sure that bugs/issues are fixed quickly and diligently.

Ask your developer to set up automatic crash-reporting inside of the application and a form that captures all of the above.

See also: Mobile App Testing | Improving Functionality & User Experience

5. Behaviour Metrics

“How do you measure user behaviour in a mobile app?”

– Janine | Marketing Consultant

Behavioural metrics are designed to measure the users behaviour outside of the realm of your mobile app. They can be broken down into the following:

• Devices & location
• Usage-time & frequency
• User actions

The metrics above are segments of general metrics that help you understand users behaviour.

Devices & location

Questions to ask yourself:

• What devices do your customers use to engage with your application?
• And where are they when they engage with your application the most?

I read an interesting case-study as I was researching for this article:

As marketers this is a significant event and we need to understand these types of ‘variables’ in our users’ behaviour.

Why didn’t she complete the purchase from her smartphone?

• Was that because the application was hard to use?
• Or because she didn’t want to get her credit card out on the bus?
• When is she making the buying decisions compared to following through with the purchase?
• How often does she need to browse before she makes a purchase?
• What does she like to browse for compared to what she actually purchases?
• Do we need to push her towards purchasing other products?
• And most importantly, is this type of behaviour demonstrated by other users as well?

By tracking the user location we can get a bearing on what her buying behaviour is, where she is when she engages with your application and what devices she prefers to use.

How would knowing the answers to the questions above change the way you build the application in future updates?

Internet Connectivity

During what times are your users not able to connect?

Do they spend most of their time outside of 3/4G zones?

If your users spend most of their time outside of internet connection zones (satellite zones) then your application needs to accommodate this. Things like this need to be coded into the application and if they aren’t already then they need to be considered for the next update.

Do they have access to high-speed Wifi?

If not then large datafile transfers are out of the question.

Usage-time & frequency

When and how frequently do your users open your application?

The average Facebook user checks their Facebook account 60 times per day. How does your application relate to that?

Actually, how do you personally relate to that?

Understanding recency & frequency metrics helps us asses how addictive/necessary our application is to our target demographic.

We also want to find out what features are most frequently visited.

These features need to be made ‘easy to reach’ and available from nearly every screen.

Why Behaviour Metrics are Important

Understanding your users behaviour gives you an insight into how your can build your application to better suit their situation.

These metrics will also change the way you promote your application and other products/services.

Putting your data into action

Once you have your reports set up you can begin to marry up important data-sets.

This is called ‘Meaningful Data’ because it points at trends, highlights problematic areas and gives you insight into key opportunities.

I’ll give you some examples of data-sets you can marry up and what insights you will get out of them:

Aquisition Metrics + Quality Metrics + Behaviour Metrics

Can tell you what types of users (based on their behaviour) are most likely to give you feedback on bugs, etc. without abandoning the application. This is useful for when introducing & testing new features in a sandbox environment with selected users.

Behaviour Metrics + Acquisition Metrics

Gives you insight into your Quality Users most important behaviours. This helps you update the User Experience to accommodate their preferences.

Retention Metrics + Quality Metrics + Acquisition Metrics

Can tell you how many of users are lost to bugs/issues/lack of features and an associated dollar value. Helps you prioritise which issues/bugs/features need to be sorted first.

Engagement Metrics + Behaviour Metrics

Shows us where users are engaging with the application and how long for. This gives you an insight into when do push a reminder notification to them. If they catch the train to work every morning then that would be a good time to engage them.

Conclusion

Clearly defined data lets you make accurate decisions. Raw, unsegmented, inaccurate data let’s you make guesses.

Your job is to set up a system that reports the data you need to know in order to make these executive decisions. And if it isn’t you who’s making the decisions then your employer/manager will love you for reporting to them with a clean spread of comprehensive metrics.

The following two tabs change content below.

• Bio
• Latest Posts

Logan Merrick

Logan Merrick is the co-founder and Director of Buzinga, as well as one of Australia's most recognised entrepreneurs, keynote speakers, investors and mentors. His writing on startups, technology and mobile marketing has been featured in The Australian, Business Insider, Startup Smart, Smart Company, and more.

Latest posts by Logan Merrick (see all)

• Ep 18: Collective Campus’ CEO on Intrapreneurship and Corporate Innovation - December 20, 2016
• 50 User Engagement Strategies For Planning Memorable Mobile Experiences - December 19, 2016
• Latest Data: App Monetisation Trends And Drivers 2015-2020 - November 25, 2016