App Store icon testing

With more than 600 000 applications, competing in every mobile application store, the icon of your application works as a mini-banner, so choosing the right icons can play a big role.

But when you look at 3 different icon variations, how to make sure which one works best? If you reduce the price of your app from 1.99 to $ 0.99, will it result in doubling the sales? Or triple? Or even more? Which option will result in best best conversion rate?

Some will conduct research, compare with competitors and bestsellers, and answer these questions based on their own ideas of beauty and intuition.

Apple's AppStore and Google Play Store do not allow to conduct A/B tests to find out which variation of icon/price/description etc., works best. So you have to resort to another method.


How to conduct a micro-test for the application

There is a better way to make these decisions. Meet - “the micro-test" - a method that will allow you to test all of the above elements: icon, price, description, and so on.

Although some large developers have been using similar techniques, the majority of developers and publishers do not even know about it.

Below we explain how to use free or inexpensive tools that can be created to run micro-tests and optimize the look of the application in the Store.

Microtesting is based on a simple idea: if we can not make A/B test in the AppStore, we can imitate it’s appearance elsewhere and thus carry out the test.


How to find the perfect icon

TapTapTap, a well-known software company, have posted the story of how they chose the wrong icon that destroyed the launch of the product.

During development, they had three options of icon design. Each team member had his own opinion, which design is best and what needs to be done to change or improve it.

App Store icon testing

Unfortunately, they did not have a set of objective facts on which to base a decision that caused a heated debate.

This story shows how, in case of insufficient user data, a "common design" has created an icon, which worked very badly. This is exactly the case when the game developers were supposed to conduct a micro-test. To help you avoid similar mistakes, we show how to create the first micro-test in 3 steps:

  • Create a landing page for A/B testing;
  • Send traffic to your landing page for testing;
  • Analyze the results.

The same steps are used to test the remaining elements (description, screenshots). On the example of an icon we will go thoroughly through all the steps, and then briefly describe the pricing testing, testing of descriptions and screenshots.

Step 1: Create a Mobile Landing Page

In our case, the Landing is the page where the user will see all of the same elements as in the mobile Store: the icon, the description, screenshots and "Buy" button. Even if the application has not yet been released to the store, you need to create a landing page, as if the application is already available. The page should contain a big button from one of the stores.

Create Landing Page

Our goal here is to imitate the availability of your app on the app store to see how many people would try to download it, depending on the icon that will be on the page.

Tools for creating a Landing Page

There are three options for creating a mobile landing page:


  • Write the code from scratch or hire a freelance developer. This is the most expensive option, but it will give more flexibility - you can change anything you’d like.
  • Use a template from a site such as Themeforest. They cost about $10 and still require some knowledge of HTML/CSS.
  • Use the constructor for mobile landing pages like Pijnz, Landr, Atmio, Convrrt or SparkPage.

In this example, we will not dwell on how to change the HTML code pages, or upload them to the server. We believe that you will be able to either figure it out.


Create Landing Page

Step 1: Set up A/B test

Now, when your mobile landing page is ready you must create multiple variations for it. Each will have its own icon variation, but all the other elements on the page should remain completely identical in all variants. In the screenshot below you can see four different landing pages, which will be used to test icons that are slightly different from each other.

Create Landing Page

The A/B test will randomly distribute the traffic between pages. This will help to determine what percentage of people on each page clicks the "Download on iTunes» button.

If you have laid out the page based on a template or from scratch, in order to create variations you can use such tools as Optimizely or Vizual Website Optimizer.

Step 2: Get users to your landing page

Once your variations will be ready and traffic distribution is set up, all that is left is the traffic. If you are popular (micro) blogger you can simply share a link to your page from Twitter, Facebook or other social media outlets. Though some find it better to buy at least a small amount of traffic in order to run the test.

If the campaign is properly set up, only $50 - $100 can result in a sufficient amount of traffic for the test.

There is another reason why the traffic is still better to buy. Optimal test includes 2-4 icons, but if you want to test more options you would need higher traffic volumes.

Traffic can be bought from AdWords, Facebook or Twitter.

traffic acquisition

Step 3: Get the results

Before your start pouring traffic to your landing pages you must configure analytics to count the number of clicks on the "Download" button. This can be done, for example, through Events in Google Analytics, send an event each time a user clicks on a button.

You can also use tools like KISSmetrics: they have the support page, where it says how to send events.

Send events to analytics

If you are not familiar with statistics, it is best to use a simple calculator that will show whether you can trust the results of the test.

Even small changes could significantly change the icon conversion rate. These additional settings can help to get into the "Top 10" mobile store or even the Editor's Choice part.

All this would increase the application’s install rate. In contrast to the intuition of the developer, the approach described above allows you to choose the best icon, based on the data. With its help, you can test and other important parts - the price, description and screenshots.

What price will give the most revenue?

Before determining the application price many developers have a discussion that may look like this:


- I'm just telling you, I'd be much more willing to buy the app that costs 99 cents rather than the $1.99 app.

- But how willing? 2 times? Three? Or one and a half?

- Dude, I do not know!


This is an important question: if you double the price, will the sales drop by 50% or more?

A simple example. If you sell the app 100 times at $0.99, you earn $99. What happens if you raise the price to $ 1.99?


  • If sales fall by 30%, you will earn $ 139.
  • If sales fall by 45%, you will earn $ 109.
  • If sales fall by 55%, you earn only $ 89.

By how much will sales fall if you double the price? This is a typical question of elasticity of demand. You are lucky that you are working on the internet and finding out the price elasticity for your application is quite simple.

Similarly to the example of the icons testing, the micro-tests can also help find the best price.

To test the price, repeat all the steps of an example with an icon. The only change - a noticeable price. Aside from the price, all the other elements in all variations must be the same. That is, it is not necessary to test both the icon and the price. First, check one, then the second.

testing the app price

Counting Cash

Once the tests start out, you can already start to figure out which option will bring the most revenue.

This can be done using KISSmetrics, SparkPage or Optimizely. Or by pulling from memory knowledge of mathematics for third grade, count yourself. The formula is very simple: the number of clicks divided by the number of visitors multiplied by 100 - get the conversion.

Despite the fact that everything seems obvious, we’d still like to provide a few examples:

Imagine that you are testing two options prices: $ 0.99 and $ 1.99. You send 1000 users to each variant of your landing page.

  • Page A ($ 0,99) gave 250 clicks on the button "Download". This provides a 25% conversion.
  • Page B ($ 1,99) gave 150 clicks on the button "Download". This gives a 15% conversion rate.

But how does this impact the revenue? The easiest way to find out - is to multiply the price of the conversion and see what happened.

  • Page A: $ 0,99 x 25% = 24.7 cents;
  • Page B: $ 1,99 x 15% = 29.8 cents;

These figures tell us that on average each user page A brings us to 24.7 cents. And each user with page B brings us to 29.8 cents.

Thus, the results of this test, the application can start to earn a 20.6% more money.

Note: These figures do not necessarily show what conversion you will actually have. It only shows the relative numbers between the two variations. This test is suitable for determining the price, but not to build any sales forecasts. Use micro testing to pick the optimal price, but not to forecast sales.


Defining the “Selling Description" for the application

Of all the metadata in the App Store description the trickiest one is the description, despite the fact that it occupies a large portion of the screen.

If you look at the popular applications you will see a huge variety of approaches to the description. Some start with advertising text, some points out user ratings and its position in the rankings. Others list all the functions, and someone picks up the top list of features from the last update.

testing the app description

As you can see, there is no single recipe for how to make a selling description and It's baffling.

Do I need to make a long description to download the app to convince anyone who wants to read it? Should it be short? Is this enough to interest users? How about a list of features? Like many they put in the description?

It is always easier to make decisions based on the data.


Running the test for the description

For this test, just do everything as written above with one exception: the landing emphasis should be placed on the text while the use of images must be minimized.

We need to check out how the description works to see it's impact on the conversion.

running tests for app description

Unlike other experiments, here the "Download" button needs to be put at the top. At first glance, it is somewhat illogical, because we want to give users the opportunity to read the description. But these things are not that simple. Since we need to see how the description in the AppStore works, it is necessary that our landing page looks as much as possible like the interface of the AppStore. Therefore the "Download" button, would have to be positioned above the description.

As in the previous tests there is no need to run too many variations of the text at once. In this case you'll have to send a lot of traffic to get statistically significant data.

Therefore, it is better to divide the whole testing process into 2 phases. For example, in one variation you highlight the high rank of the application, in the second - the list of functionality and on the third one - the eye-catching description.

After testing to determine the best style, proceed to Phase 2 - Check the various versions of the texts in this style. For example, if your application is worksheets with the list of functionality, try to arrange the list in a different order or make it different lengths.


Choosing screenshots for applications

We hope that by this step you have already begun to identify patterns of how to run the tests.

Screenshots are often underestimated in terms of their effect on the installation. But it is a screenshot that a user sees when your application appears in the search results. And that it takes in the AppStore starting with iOS 7 the most prominent part of the application.

Testing screenshots is no different from the examples above, except for one thing. Since you would want to copy the entire interface of the AppStore, it is best to implement a preview screenshots by swipe.

running tests for app screenshots


Conclusion

These tips will give you a basic understanding of how to test the elements that cannot be tested in the real environment of the App Stores. This is an incredibly cool tool for improving conversions.

Our G+