True Impact

Overview

True Impact is a reporting tool that shows you the results of your Localytics messages. To access the report for one of your messages, head to the Messaging screen in the Localytics dashboard and click on the message's name.

Along with displaying basic metrics like sends, opens, and conversions, True Impact also gives you additional insights into how your messages drive changes in user behavior.

Some use cases might be:

• Consume at-a-glance Hero insights around specific message performance.
• See positive as well as adverse downstream effects of your message.
• Dive into deeper insights to see other commonly observed user behaviors.

Each True Impact report will be personalized your message type, goal, length, and user experience—so every True Impact report is different. There are a few metrics that span across each of the message types—so let's dive into the different parts of a True Impact report.

If you need to Edit, Duplicate, or Archive your campaign, click the three dots at the top of the page. Here you can also quickly reference the Campaign ID if you need it.

Message Summary

The top part of your True Impact report is what we call the message summary. Here you'll find high-level information about your message, as well as see some hero insights and a preview of the message.

Let's dive into each.

Message Information

At the top of the True Impact report is some top-level information about your message.

Here's a quick explanation of each of these terms:

• Channel - the type of message: push, in-app, inbox
• Goal - the goal you set during the building phase: activate, drive behavior, nurture, monetize, re-engage, notify (hover over for more information)
• Conversion event - the event which measures the success of the goal (if you've applied a filter, you'll see a filter icon here as well—hover over for more information)
• Control group - if you enabled a control group, you'll see the % here
• Audience - which audience you chose to send the message to (we'll only show a name here if you've chosen from a saved Audience—if you built one in the message builder, you'll see it listed as "custom")
• When - the date the campaign was launched and potentially finished (hover over the icons for more information)
• Status - active or expired

Hero Insight

Below the message information is a Hero insight, which gives you some insight into how successful your message has been.

The information shown in the Hero insight is determined by the Goal for the message, which you specified when you built your message. If the Goal of your message was Monetize then the Hero insight would show you details around how the message is driving changes in Revenue Per User.

Depending on the performance of your message, this metric (and all other metrics in True Impact) will be red, green, or gray. These colors indicate statistically significant negative change, statistically significant positive change, and no significant change, respectively.

If it has been less than 48 hours since your message went live, you might not see a Hero insight. Don’t worry, everything is still working properly! Check out the section on Live Results below for more information.

Message Preview

Next to the Hero insight is a preview of your message. If you are running an A/B/n test, you can view each message version by clicking on the letters above the preview. If you hover your mouse over a letter, you can see what percentage of the audience received that message.

Navigation Bar

Below the message information is a navigation bar that lets you move between the three sections of True Impact: Overview, Insights, and A/B tests (we'll talk about Explore in a later section).

We'll explain each of the sections a bit more in-depth in the next part of this article, but at a high-level, here's what you'll find.

Let's start with the Overview section.

Overview Section

This section gives you a high-level summary of how your message is performing.

Here you'll see insights related to the Goal you chose—things like conversion rate, revenue per user, and uninstalls. Again, keep in mind the colors (green for positive change, red for negative, and gray for no change).

The Overview section consolidates all the information marketers care about most into one place. This allows you to quickly and easily check on the health of your message. The following sections will explain the components of the Overview screen.

Key Insights

In Key Insights, you will find several metrics that give you an idea of how your message has driven changes in user behavior. Hover over any of the numbers for a deeper explanation.

The metrics shown depend on the Goal of your message. If your message's goal was Monetize then you will find Revenue per User and Number of Paying Users as two of the metrics displayed in Key Insights.

The metric shown on the right is whichever metric was most negatively impacted by the message. It highlights any unintended consequences of the message.

Live Results vs. Full Campaign

Below Key Insights is a selector that allows you to toggle between Live Results and Full Campaign.

This selector determines what data is shown in Stats and Trends, which are the two remaining components of the Overview section.

If you select Live Results, you will be shown data from the past 48 hours, updated every five minutes. In contrast, if you select Full Campaign, you will see more metrics, but they are only updated every 24 hours.

Live Results is a great way to take a look at the initial performance of a message in the minutes and hours after it goes live, while Full Campaign is the better choice for getting an overall sense of how your message is doing.

Stats

In Stats, you will find a variety of standard message metrics such as sends, clicks, and conversions. Stats is where you should turn to get a sense of how your message is performing. Heads up—this section will be labeled Recent Activity if you're viewing the live results.

Trends

Trends gives you a sense of what’s been happening with your message over time. It shows you how sends, opens, and conversions have changed over time. Hovering over different parts of the chart will give you more insights—like isolating particular metrics, or viewing the exact number of sends on a given day.

Insights Section

The Insights section helps you understand the details of how your message changed user behavior. The metrics in the Insights section are split into three groups: Engagement, Conversion, and Revenue.

Engagement

Here you'll find metrics around how users interacted with your app in the days and weeks after they received your message. Metrics like average session length, retention rate, push opt-in/opt-out rates, and uninstall rates are all found here.

Filter any of these metrics by creative (if you decided to use the A/B testing functionality) and date. Just select the option from the dropdown menu.

Conversion

If you set a conversion event when you were building your message (and you should always set a conversion event), this is where you’ll get details around how many members of the message's Audience performed the conversion event, if they performed the event multiple times, and much more.

Revenue

If you have configured your event tracking to capture LTV, you’ll find detailed metrics on how your message drove changes in the LTV of users who received your messages.

A/B Test Section

If your message has multiple creatives, you have the ability to compare each version of the message side-by-side in the A/B Test section.

See how your creatives helped drive key metrics associated with the Goal for your message. You can also uncover which creatives are leading to adverse effects downstream. See how each creative performed against a control group. Use these insights to adjust the distribution of creatives to reflect the most successful version.

For a definition of any of the metrics you might see, check out the chart below.

* For In-App messages, an Open is any "Localytics In-App Displayed" event that comes through with an "Action" attribute that is not equal to "X."  For Inbox messages, an Open is any "Localytics Inbox Message Viewed" event that comes through with an "Action" attribute that is not equal to "X."

Explore

While you hope your users perform the conversion event in your message, you may want to see if your message has caused users to perform other actions as well. The Explore tab lets you uncover insights for users who received your message and performed different behaviors.

View other insights by specifying a different conversion event associated with message success. Alternatively, you could diagnose poor side effects of your message by selecting an event that could be considered the opposite of your conversion event.

With Explore, you can see conversion rates for messages compared to the control group with four different time periods to choose from (1, 3, 7, and 14-day window). Toggle between the options to see the different time periods.

Explore also allows you to hone in on a subset of users by applying a filter to your conversion event. These filters can contain event attributes as well as custom dimensions. Use the + Filter option to add a filter.