Mobile Testing Challenges – Surviving the Mad App Rush

Mobile Testing Challenges – Surviving the Mad App Rush

The rising popularity of smartphones presents a wealth of opportunities for app developers and publishers. As with most opportunities, there is no shortage of challenges. We’ve noted some of them below.

The primary challenge and the one we will go into the most in this post, is testing these apps, so that they work in all your target market environments.

Now, while a tester who is used to testing web applications gives them a leg up on other testers with no experience, mobile testing is a completely different ball game. In the web world, a tester might have to test a handful of web browsers. When the complexity of different devices, operating systems, and models is introduced, this creates a challenge of paramount proportions.

This is especially the case with the Android and Windows mobile platforms, where companies such as Samsung and Nokia dominate the market. We are not just talking about different devices. We are talking about the multiple operating systems that are resident on the devices of your target market.

One way, which is highly common in testing the array of devices, is by using device emulation. While emulation is less expensive, there is a reason for it. Nothing beats the real thing. With emulation there is always the high chance that your testing might not come close to a real life environment that is needed to test.

Research. Research. Research.

One way to cut down this hassle is to research your target market. Say, you are releasing an app to the Swedish market that delivers meatballs to you door, hot and ready to eat.

You discover that by conducting some research that Swedish mobile users mostly have Samsung and Nokia issued devices. In this case, you should test on the most used device models, indicating it in the app store description. You might want to say that specific devices have been tested and more are being tested. You cannot guarantee the successful use of the app on phones not specified.

Another important thing you should research are the most commonly used network protocols. You wouldn’t want to have built your app with the 3G network in mind for connection, when 3G doesn’t even exist in one of your primary target markets.

The Mad App Rush

Because bringing an app to market is almost always based on timing, there are mad rushes to the app finish line. With this release frequency, comes the heightened confusion regarding testing among the team, which could lead to a buggy release if things are not done precisely. Remember, there is no room for error. If you release a buggy app, you might just get booted out of the app store where you are publishing.

Additionally, functional testing isn’t the only type of testing you should be conducting. After the app is launched, utilizing an in-app analytics tool is key to optimizing the mobile user experience.

Remember, you only have one chance to make a first impression. This statement looms large over the mobile app sphere. If your app doesn’t work upon launch, as the user perceives it should, you may have just lost a customer, never to return.

 

Mobile App QA – Should you automate?

Mobile App QA – Should you automate?

Mobile apps have completely changed the way we live our lives. With the touch of a button, I can find a restaurant, get hotel reviews, get directions, or order a cab. Long gone are the days when I printed out my MapQuest directions and studied them whenever I needed to go somewhere new. While I enjoy how easily information is available to me now through my smartphone, I know it would not be possible if businesses didn’t keep up. Thankfully they are!

Businesses are realizing more and more that mobile apps are the best way to reach the highest number of customers. As more apps make their way into the market, the need for QA and testing also increases. With multiple platforms and ever progressing features, this isn’t such an easy task. That’s why we recommend QA automation! I don’t expect you to take my word for it without any facts to back it up so let’s dive in.

Why automate QA and testing?

Mobile app testing is often very repetitive, tedious, and time-consuming. With automation, you can simplify your process by defining a test once that can be executed multiple times. This reduces your time investment, cost, and error margin while increasing your productivity and test coverage. It also enables you to create a more stable application by making it easier to run tests after every minor change to the code. Essentially, automation will increase your productivity, ensure an agile development team, and reduce overall costs.

This is all very nice but the world is unfortunately not so black and white. While the need for mobile app QA is significant, there are many common problems that QA teams encounter that can disrupt and limit the efficient testing of enterprise apps. Some of the problems that enterprise testers come across are:

  1. Mobile app testing is not thorough – or not done at all
  2. Lags in development can lead to shorter testing time
  3. Lack of requirements can inhibit QA capabilities

To read more about the problems QA folk come across, check out this article: Three Common Problems in the Mobile APP QA Process

Mobile Testing & QA Tools – Should I be using automation?

Mobile Testing & QA Tools

mobile-headerI am not going to answer the above question as there is not right answer but hope to have a conversation that will help answer for those asking. With the smartphone revolution upon us and the multitude of applications being released on a daily basis there is an increasing need for tools and automation to test all of these mobile applications. We are partnering and continuously vetting out tools for our customers.  One thing I can see now after 8 years in this space is that there is not one tool that is perfect or that can be called the best.  This is because it highly depends on the application.

Some questions are:

  • What parts of the app can be tested through automation?
  • Is location relevant?
  • 2G, 3G, Wifi, LTE or is connectivity available?
  • Does usability need to be tested?
  • Is this an enterprise or consumer app
  • How important is security
  • Does the backend need to be stress tested?

These are only some questions – please feel free to add more.

I have seen the tools fall into 3 categories with each one having a multitude of solutions:

  • Crowd Testing
  • Remote access to devices
  • Mobile QA / Test automation

Mobile Testing – Automation or Manual

Mobile Testing – Automation or Manual

Posted by Perry Nalevka on 

mobile-application-testing1-300x222

In previous posts we wrote that there is not enough testing being done in the mobile space but we see that the trend is changing as more outsource vendors (like us) are offering Mobile QA as a service.  The next problem I am seeing is that most testing being done today is manual which creates a lot of delay in today’s agile development world.  Although some manual testing will always be needed there is definitely room for more automation.  What’s driving more a more companies toward test automation is also the ever shrinking release cycle they’re facing for their mobile apps. Whether it’s a mobile web app which can see multiple versions per day, or a native/hybrid app which can be released any other week, the need for continuous feedback is critical.