User experience optimization

Common questions from developers using MobileEngine are about how to best use the API from a user experience prospective. We have prepared the following guidelines to help.

Mobile apps are all about quick and easy results with as little thinking as possible. For an app with a tight focus, ease of use comes from having as few interactions and as few choices as possible. Once the app has been told what to do, user satisfaction comes from immediate feedback and results. Every quarter-second counts!

Ease of use

To make your app quick and simple to use there should be as few screen taps as possible to perform a search. Ideally, the user would do

  • One tap to open the app. If your app’s main focus is recognizing items using image recognition, then your app can launch directly into camera mode.
  • One tap to start a search. We recommend that this tap would launch camera mode, without first asking if the user wants to use an existing photo. If using an existing photo is possible, add an icon to do this that is visible from your camera-mode screen.
  • One tap to take the photo. We recommend that you perform the search immediately at this point, without having a “preview & use” step. Users quickly learn to frame shots well, and the extra step just slows them down. Likewise we do not recommend a step to allow user to scale and crop the image.
  • While your app is sending the photo, show the user the photo they took while updating them on the progress of the search. E.g. “Sending photo...” “Recognizing image...”
  • Then display the best match identified by MobileEngine, or another action if the object is not recognized.

In addition, the first time a user uses your app, show them a simple one page guide on how to take a good photo with an example. Only show this page again if the user requests it.


Query images

There are two main parts to the time it takes to get results: sending the query image over WiFi/3G/4G/LTE, etc., and performing the search in MobleEngine. Both of these are affected by the size of the query image. Large query images are scaled to be 600 pixels in the smallest dimension when they are received. Scaling them to exactly that size before sending will save transmission time, and also saves the time needed to do the scaling after receipt.

Collection images

The images you add to your collection should be 600 pixels or larger in the smallest dimension. Larger images will not make for better matches, since they are scaled down before indexing. Because adding collection images is usually less time critical then searching for images, it is typically less important to pre-scale images to exactly this size before giving the ADD command.

Your images should concentrate on the main subject and avoid background clutter, and even less-important parts of a product. For example, if you are matching beverage labels, make the label fill the frame as much as possible, rather than shooting the whole beverage container. If possible, shoot against a uniform blank background. Objects should be shot straight-on and upright.