Mobile Testing & QA Tools – Should I be using automation?

Mobile Testing & QA Tools

mobile-headerI am not going to answer the above question as there is not right answer but hope to have a conversation that will help answer for those asking. With the smartphone revolution upon us and the multitude of applications being released on a daily basis there is an increasing need for tools and automation to test all of these mobile applications. We are partnering and continuously vetting out tools for our customers.  One thing I can see now after 8 years in this space is that there is not one tool that is perfect or that can be called the best.  This is because it highly depends on the application.

Some questions are:

  • What parts of the app can be tested through automation?
  • Is location relevant?
  • 2G, 3G, Wifi, LTE or is connectivity available?
  • Does usability need to be tested?
  • Is this an enterprise or consumer app
  • How important is security
  • Does the backend need to be stress tested?

These are only some questions – please feel free to add more.

I have seen the tools fall into 3 categories with each one having a multitude of solutions:

  • Crowd Testing
  • Remote access to devices
  • Mobile QA / Test automation

Mobile Testing – Automation or Manual

Mobile Testing – Automation or Manual

Posted by Perry Nalevka on 


In previous posts we wrote that there is not enough testing being done in the mobile space but we see that the trend is changing as more outsource vendors (like us) are offering Mobile QA as a service.  The next problem I am seeing is that most testing being done today is manual which creates a lot of delay in today’s agile development world.  Although some manual testing will always be needed there is definitely room for more automation.  What’s driving more a more companies toward test automation is also the ever shrinking release cycle they’re facing for their mobile apps. Whether it’s a mobile web app which can see multiple versions per day, or a native/hybrid app which can be released any other week, the need for continuous feedback is critical.


World Quality Report Reveals that Two-thirds of Companies Are Inadequately Testing Mobile Applications

World Quality Report Reveals that Two-thirds of Companies Are Inadequately Testing Mobile Applications
World Quality Report 2012-2013 just revealed some major factors about Mobile App Testing few days back. As mobile devices continue to transform users’ needs, demands and expectations, the report highlights the need for companies to rethink their QA strategies and priorities.The report revealed some of the important facts in Mobile Testing on the basis of responses of the respondents who participated in to this.The report is elaborated on Mobile Testing and other part of Software Testing. The report revealed that  Organizations may not be giving mobility the priority it deserves. This sector is still facing some challenges due to lack of expertise resources,best Practices, availability of devices etc. Performance of Mobile Apps emerged as the top most focus for most of the firms who participated in this survey.
Though there are many points discussed in the report about Mobile App Testing, here are some important points that came out of this report.
  • Two-thirds of companies are inadequately testing mobile applications
  • Only 31% of respondents currently test mobile applications
  • 65% of respondents conveyed that they don’t have right tools to test mobile apps.
  • 52% do not have access to required devices.
  • 34% of organizations lack the testing methodologies and processes
  • 29% of organization lacks specialist expertise  necessary to effectively certify mobile application
  • Performance of the mobile app has emerged to be primary focus for most of the organizations( 64%)
  • 37%  are Testing Mobile App in Manufacturing domain followed by Energy& Utilities and Healthcare(Each 35%)

Picture speaks louder than words so here are some quick trends that came out of this report.



 Details taken from –

Three Common Problems in the Mobile App QA Process

Three Common Problems in the Mobile App QA Process December 19, 2012

mobile-app-qa-300x214Quality assurance (QA) is an essential part of creating and maintaining reliable mobile apps that function as planned – and the need for these apps in the enterprise is increasing exponentially. Businesses of all sizes, enterprise-sized companies in particular, are developing new mobile apps as well as converting business critical Web and desktop apps into mobile apps to improve employee efficiency and productivity and provide more responsive customer service.

Coinciding with this increase in need is an even faster evolution of operating systems on computers and especially mobile devices. This presents testers of all types – including those testing native and mobile web apps – with the challenge of delivering apps that function well on multiple platforms, devices and form factors.

While the need for mobile app QA is significant, there are also many common problems that QA teams encounter that can disrupt and limit the efficient testing of enterprise apps. Here are three common problems that enterprise testers may encounter.

Problem #1: Mobile App Testing is Not Thorough – or Not Completed at All

QA has become an increasingly important part of software development. However, a recent survey conducted by Cap Gemini, Sogeti and HP shows that QA for mobile apps is not enjoying the same prominence. In fact, the survey shows that only 31 percent of organizations today currently test their mobile applications. This means that many enterprises are risking the release of apps that don’t function properly, are unreliable, unstable and potentially damaging to their business functions and their brand’s reputation.

Companies all over the world are trying to save time, increase profits and decrease risk. The process of testing native and mobile web apps may decrease risk, but it also takes time to complete – time that many companies are not willing to spend if it means extending app deployment deadlines. It’s a delicate balance that too often ends up with mobile app testing being pushed to the side as apps are rushed to be released.

Problem #2 – Lags in Development Can Lead to Shorter Testing Time

QA is one of the final steps in the development process before an app is released. Before an app makes it to QA for functional and regression testing – not to mention usability, performance and security testing – there are several steps that must first take place, including all phases of development, such as technical design, wireframe development and mobile app development for each mobile platform. As such, if there are lags in the development of an app, testers will have to deal with a shorter than average test time, often resulting in rushed functional testing and running out of time to regression test.

The testing cycle for mobile apps normally takes three-to-six weeks depending on the size and complexity of the application. Due to expedited enterprise app rollouts, this is now being condensed to just one-to-two days, which hardly leaves time for proper testing.

Problem #3 – Lack of Requirements Can Inhibit QA Capabilities

Good requirements from the beginning – even before development starts – can help with the development and subsequent testing processes. Unfortunately, in many cases, mobile app testers aren’t receiving these requirements. This is a critical misstep since providing testers with the requirements can help them effectively define what they should test and how they should develop the testing framework. Without the requirements, it’s like testing in the dark. The key to efficient and accurate test results is to enable testers to plan and test better – and they need requirements to do so.

Visibility to app requirements can also help testers determine whether parts of the testing process can be automated. When automation makes sense, testing cycles can be significantly reduced, freeing up testers to focus on more complicated test cases that either cannot be automated or require more detailed scripting at the outset. Enterprise QA processes are undergoing a major re-evaluation and re-invention thanks to the mobile revolution. By planning ahead, using a mobile app test automation tool and emphasizing the importance of QA to the overall effectiveness of an app, QA teams can avoid these problems and improve overall app quality and reliability.

This article, Three Common Problems in the Mobile App QA Process, was based on an article on MobileLabs

The Globe-Trotting iPhone 5

The Globe-Trotting iPhone 5

In September, app developers were scrambling to optimize their apps for the new iPhone 5 -making sure apps looked right and took advantage of the phone’s larger screen and enhanced features.

Now, those who want a global presence will have to make sure their apps are ready for the iPhone 5 around the world – meaning localization testing will be in order. The latest iPhone will be released in South Korea this Friday, it will hit more new countries next week and the final wave will come later this month – more than 50 countries in all.

Here are the countries you need to prepare for:

Dec. 14

Albania, Antigua and Barbuda, Armenia, Bahamas, Bahrain, Bolivia, Brazil, Chile, China, Costa Rica, Cyprus, Ecuador, Grenada, Indonesia, Israel, Jamaica, Jordan, Kuwait, Macedonia, Malaysia, Moldova, Montenegro, Panama, Paraguay, Philippines, Qatar, Russia, Saudi Arabia, South Africa, Taiwan, Turkey, United Arab Emirates and Venezuela.

Dec. 21

Barbados, Botswana, Cameroon, Central African Republic, Egypt, Guinea, Ivory Coast, Kenya, Madagascar, Mali, Mauritius, Morocco, Niger, Senegal, St. Kitts, St. Lucia, St.Vincent & the Grenadines, Tunisia, Uganda and Vietnam.

Thanks to engadget for the information. They have the full press release on the site if you’re interested.

Posted on 12/06/2012 in iPhone App Testing by Jamie Saine

Do We Worry About App Security Enough?

Do We Worry About App Security Enough?

Do we worry about security enough? Does it keep us up late at night, constantly in the back of our minds? If not, it should.

The reality is, as developers and app users we hardly focus on security enough. Jon Evans of TechCrunch says as users instead of worrying about it ourselves, we let the Facebooks and Googles of the world take care of it for us – which puts both us and the companies and apps we rely on in some treacherous territories:

“Alas, right now it seems that many-to-most people value conformity more than privacy.What’s more, instead of worrying about security ourselves, we trust others — Amazon, Apple, Facebook, Google — to take care of it for us. As the great Bruce Schneier points out, in some ways we’ve regressed to a feudal notion of security.

… Security is, by its very nature, something most people generally hardly worry about at all – until and unless that one awful day comes when it’s the only thing they worry about. By then it’s usually too late to start taking it seriously. “

As users we can certainly become more security-savvy. But more importantly, as app developers, security needs to be top of mind.  Security failures usually occur because of poor design and a lack of testing. Therefore, if there was more forward thinking during the design phase developers would be able to produce better, more secure apps.

As T.L. Neff of Wired says, when it comes to development and security “less is more” and forward thinking is essential:

“Overall, users must include security factors while designing the app. Sure, you can be conservative about what you expose in the first place. Definitely consider some limits on what can be downloaded, and think about using graphical cues instead of text. By taking these kinds of steps, you’ll likely end up with apps that are more streamlined and user-friendly, and minimize security risks for your company.

The bottom line: don’t approach security as a set of utilities you put in place after apps are deployed. You’ll get better security through more of a life-cycle approach where you design with security in mind, and also test for security.”

It seems like it will take a collective effort from companies, developers and users alike in order to really improve mobile app security. Looking for resources on mobile app security? Here is a free whitepaper with security testing tips on common attacks, security tools and ways to build a better QA team: Security Testing.

Posted on 12/04/2012 in Security Testing by Katherine Slattery

How Many Apps?

How Many Apps?

There’s sometimes a question as to how many apps an organisation/company should produce. Should you bundle all required functionality into one app or produce separate apps?

The overriding consideration is obviously cost. It costs a lot more to produce, test, support and distribute multiple apps rather than have everything on one app. So what considerations might make the added cost worth it? Here are some scenarios I have come across that have resulted in multiple apps…

  • Where having multiple apps will lead to more overall engagement. For example, an app used primarily for marketing might have functionality spread over several apps, over time, so as to spread and increase engagement over time.
  • When it’s known all the functionality won’t be ready all at once and it’s technically difficult to upgrade older functionality if it were all in one upgraded app.
  • When the functionality is totally separate and we don’t want users to know or care about functionality they won’t even use. e.g. Partitioning apps by enterprise job function (or this can also be done with one app by only showing functionality applicable to the user).
  • When the user experience for given functionality needs to be very different and placing all in one app would produce a confused app and confused users.
  • When functionality is monetised differently (e.g. Free vs Paid). A free app is sometimes used as a lead-in for a paid app even though the functionality of the two are different, yet related. e.g. A PDF reader app might be a free app that leads to a paid app that scans things and produces PDFs.
  • When it’s easier to monetise incrementally per app rather than multiple in-app payment. For example, content is often less visible embedded with other content within an app but it might be more visible if presented as an app in its own right. e.g. People might search for book topics, not a book reader.


This entry was posted on Monday, November 19th, 2012 at 10:16 am by Simon Judge –