App Store screenshots
A/B testing App Store screenshots without guessing
A screenshot test answers one narrow question: does this set of panels convert better than the set on the page now? App Store Connect can run that test for you through Product Page Optimization, and it reports confidence instead of a winner. Starting the test is easy. Keeping it honest enough that the result still means something in week six is the part that takes care.
By Keymaker Technologies LLC · Published
One variable per test
A treatment can change the app icon, the screenshots and the app previews. That is a wide brush, and a wide brush produces a result nobody can attribute. If all three change and conversion moves, the page changed; which element did the work stays unknown.
Screenshot order is the cheapest first test because it needs no new artwork. Swap panels two and three, leave every other field alone, and the result speaks to one decision. Headline copy is a reasonable second test. Changing five captions at once returns to the same fog.
Write the hypothesis in one sentence and give it a direction: "moving the shared-document panel ahead of the editor panel will raise conversion on the 6.9-inch set". A hypothesis with no direction turns any result into a story that fits.
Where the test can run
Three conditions decide whether a test is available at all. The app has to be Ready for Distribution. The test metadata has to pass App Review before the test starts, which you can submit without a new binary. And treatments reach people on iOS 15 and iPadOS 15 or later, so a device still on iOS 14 sees the original page.
Each treatment begins as a copy of the live product page. Leave a device size untouched and that size keeps serving the original screenshots, which is useful: test the 6.9-inch set and leave the iPad set alone instead of spending traffic on a second question. The setup steps live in the [Product Page Optimization help](https://developer.apple.com/help/app-store-connect/create-product-page-optimization-tests/create-a-test).
A traffic split that can finish
Apple asks for a traffic proportion, meaning the share of people randomly shown a treatment instead of the original page. The math is literal. Three treatments at 30% puts 10% of total traffic on each one while the original keeps 70%.
That split decides how long the test needs. Apple estimates duration from daily impressions and new downloads, and the estimate is a guide rather than a promise. On modest traffic, three treatments can run the full 90 days and end inconclusive. Two treatments at 50% put a quarter of traffic on each variant, which is a blunter comparison but one that reaches a verdict.
A test runs for 90 days or until you stop it, and stopping is final: the same test cannot be restarted. Treat the split as a budget you are spending.
Make the difference readable at thumbnail size
Apple allows at most ten screenshots for each device size, and the first one to three appear in search results when there is no app preview. The difference under test therefore has to be visible in a panel read at around 200 pixels wide, in sequence, without the description.
A swapped panel changes what a browser reads second. A new font weight or a moved badge does not, and a test built on that kind of difference measures noise. Keep the device frame, the background and the type scale fixed, then change the idea inside the frame.
Export at the exact store size. The 6.9-inch canvas measures 1320 × 2868 px, and an upload that drifts off the slot is rejected before the test can use it. Orientation matters too, since a portrait set and a landscape set truncate differently. The screenshot size guide lists every accepted slot, and the preview simulator renders the search truncation without an upload.
Read the confidence level, not the percentage
App Analytics starts reporting a test once five first-time downloads are attributed to it, and labels a treatment Performing Better or Performing Worse only at 90% confidence. Before that, the panel says data is still collecting.
The label people misread is likely to be inconclusive. It means the current traffic pattern probably will not settle the question within 90 days. The fix is a larger change or a busier window, not an early stop. Ending a test because one variant leads in week two throws away the mechanism built to prevent that call.
Applying a treatment ends the test and copies its screenshots and previews onto the original page. Apple documents that the action cannot be undone, and that only previews and screenshots transfer, so a tested icon stays where it is. Keep the winning export in the same project that produced the variant, or the applied page drifts from the tested one.
Google Play runs the same discipline
Play Console separates graphics experiments from localized ones. One default graphics experiment can run at a time, alongside up to five localized experiments, and each holds at most two variants against the current listing. The controls are explicit: audience percentage, minimum detectable effect, confidence level, and a target metric chosen from unique user install clicks, open clicks or pre-registration clicks.
The discipline carries over. Test one asset at a time, as the [Play Console help](https://support.google.com/googleplay/android-developer/answer/12053285) recommends, or the result cannot be attributed. Experiments stop on their own after six months, and more data needed is a normal outcome for a small app. Neither store will rescue a test set up to measure three things.
Log the test while it runs
A log with six fields covers most of it: hypothesis, panels changed, traffic split, start date, target metric, confidence at the end. Add what you applied. The log stops a team from re-running a rejected idea a year later, and it turns a series of tests into a baseline.
Screenshot order, caption length and proof placement are the levers a listing usually has. Test them one at a time. A post on how to order App Store screenshots covers the sequence before a test exists, and the browser studio keeps each variant as a separate project so the applied set matches the export.
Questions
Common questions.
How many treatments can one App Store test have?
Up to three treatments per test, per the App Store Connect help. Each treatment is a full copy of the product page, so more treatments split the same traffic into smaller slices.
How long does a screenshot test take?
The test runs for 90 days or until you stop it. You need at least five first-time downloads before results appear, and 90% confidence before a treatment is labeled better or worse. Low-traffic apps often run the whole 90 days.
Can I test screenshots without shipping a new build?
Yes. Screenshots and previews are metadata, and test metadata can go to App Review without a new app version. An alternate app icon is the exception: any icon used in a treatment has to be inside the binary already on the store.
Does Google Play have the same screenshot test?
Yes, as a store listing experiment. You can run one default graphics experiment plus up to five localized ones, with at most two variants each. The experiment stops on its own after six months.
Where to go next
Build the sequence once.
Export it at every store size.
Compose every panel in one editable project, then export the exact presets App Store Connect expects.