An Overview of SoapUI Testing Tool

SoapUI is a free open source, cross-platform API testing tool, mainly used to test Web services and Web APIs. The SoapUI testing tool’s functionality covers features like web service inspection, invoking, development, simulation, and mocking; and can automate functional, load, regression, and compliance testing. It has an inbuilt reporting tool and ‘export test results log’ provision.

Built entirely on Java Platform, SoapUI uses Java Swing for the user interface and is simple to install. Moreover, it offers support for standard protocols and technologies to test APIs. It supports third-party plugins and is easy to integrate with IDEs like NetBeans, Eclipse, and Maven. Developers can also build custom plugins using SoapUI APIs. The protocols supported by SoapUI can be represented as follows:

With a simple interface, SoapUI allows even the non-technical users to use the tool flawlessly. Let’s discuss some of the commonly used features of SOAP UI testing.

Functional testing – You don’t have to be a developer to write automated functional tests in SoapUI. It’s amazingly simple. With the SoapUI testing tool, testers can leverage the following features:

a. Drag and Drop Test Creation
b. Testing Complex Scenarios
c. Creating Custom Scripts
d. Multi-Environment Support
e. Asynchronous Testing Support
f. Test Debugging

Compliance Testing SoapUI allows testers to validate the compliance tests and automate those test scripts.

Security testing –SoapUI testing tool supports security testing to ensure authorization and authenticity in request and response model of the Web services and Web APIs. In the context of security, following features can be leveraged by testers:

a. Cross -Site scripting scans
b. XML Bomb Scans for Stack Overflows
c. Scans to identify erratic service behavior
d. Database security via  scans for potential SQL injections

Load Testing – It helps analyze the load that a web service or Web API can withhold before it breaks. For this type of testing, SoapUI testing tool supports the following features:

a. Real-time Interactive Distributed Load Testing
b. Drag and Drop Test Creation
c. Access to pre-built load strategies to test service performance
d. Performance Monitoring
e. Auto-generated and custom reporting

Regression Testing This is conducted to analyze any malfunctioning within the web services when any new change has been introduced. Testers can execute test suites using SoapUI to find out if the existing functionalities are working as expected.

SoapUI Integration

SoapUI can be integrated with a number of tools. Some of the popular ones are Maven, Hudson, JUnit and Apache Ant.

SoapUI Drawbacks

Despite the above advantages, there are a few drawbacks of the SoapUI testing tool:

a. SoapUI lacks adequate documentation when using advanced features.
b. Load Testing and workflow testing are not easy to perform in SoapUI.
c. Using SoapUI, memory consumption is a bit higher in concurrent testing.

Conclusion

Though SOAPUI is a widely used cross-platform open source testing tool for Web API or web services testing, it is not entirely free from drawbacks. However, with easy to use interface and enterprise-class features, it is here to stay for long. A professional version of SoapUI called SoapUI Pro is also available for advanced SOAP and REST API functional testing, and offers more features along with making test creation simpler.

For testers, this implies that learning this tool is vital for them to remain competitive in the market. However, to stay ahead of the game, testers need to continuously learn new and upcoming tools and skills, stay up-to-date and take relevant certifications. Happy learning!

 

6 Mobile Test Automation Tools You Should Know About

The user’s threshold to faulty products has surely gone way below the desktop era, and advancement in technology has led to increasingly complex app functionalities putting tester’s job on the spot. On one hand, comprehensive testing solutions are required to ensure an error free end product, and on the other, time has become a scarce resource in this competitive environment. Automation has thus acquired a significant place in the testing process. Although manual component cannot be overlooked or done away with, however, deploying the right mobile test automation tools, where feasible, can reduce the costs and speed up the delivery process.

Let’s observe some of the popular mobile test automation frameworks.

Mobile Test Automation Trends

According to Indiumsoft, some of the popular Mobile Test Automation Tools and Frameworks are as follows:

Source: Indiumsoft

Appium, Calabash and Espresso are popular choices, with possibilities of XCTest and XCUITest turning out to be the next big frameworks in the iOS test automation.

In tandem with these popular choices, let us focus on the main mobile automation testing tools, the advantages they offer, along with some of their limitations.

6  Popular Mobile Test Automation Tools

Appium – It is an open source free mobile automation tool offering automation of native and hybrid mobile apps. Appium is written using the JSON wire protocol and internally interacts with apps using Selenium WebDriver and Node.js.

Some advantages of using the Appium testing framework:

  • First, it’s a free to use open source tool used for automating both Android and iOS platform apps.
  • Second, it supports a number of programming languages such as Java, C#, and Ruby without the need to modify apps for automation purposes.
  • Third, Appium is a ‘cross-platform’ tool enabling substantial code reuse between iOS and Android test suites.
  • Finally, it runs on multiple devices and emulators offering scalability.

The downside is that it offers no support for toast messages and has slow scripting. Another weakness is reduced XPath support on mobile devices. Additionally, some testing professionals feel that the test reports generated are poor and inadequate.

Calabash – It is yet another free open source mobile automation tool for Android and iOS mobile apps. Calabash testing framework allows you to write and execute tests that validate the functionality of iOS and Android Apps. It is written in Ruby.

Some advantages of using Calabash testing framework:

  • First, it is a free open source tool used for automating and testing any Android and iOS platform apps, native or hybrid, from the UI level down.
  • Second, it’s cross platform tool.
  • Third, Calabash supports testing on real devices as well as on simulators.
  • Fourth, Calabash allows support for complex features as gestures — like tap, swipe and rotate, and generate better reports than Appium.
  • Finally, with Cucumber framework (BDD), feature files can be easily written by QA, domain expert, or by developers.

Some of the cons include lack of adequate online support, non-availability of an IDE or an Editor, the need for Calabash framework to be instrumented inside ipa file for iOS, and unfriendliness to languages other than Ruby.

Espresso – It is Google’s open source test automation framework for native Android apps, and is quite suitable for white-box testing. Espresso is inside the application and has been written by Google primarily for functional testing of an Android UI. It is mainly targeted for developers.

Some of the advantages of using Espresso testing framework:

  • First, the tool is simple to setup and supports all android versions.
  • Second, it’s library is supported and maintained by Google community.
  • Third, Espresso does not need a server to communicate with, instead, it runs simultaneously with the app and delivers quick test results to the developer.
  • Fourth, Espresso allows for less testing flakiness and better test cycle stability with its unique synchronization method.
  • Finally, with Espresso, extensive testing is possible as it covers a number of UI actions and gestures.

Some of the cons include lack of cross platform support, and besides, it is not a dev-language free framework like Appium and is an app-context only tool.

UIAutomator – It is an open source testing framework by Google that offers advanced UI testing of native Android apps and games. It uses Java as the programming language and is considered as one of the best tools available for Android apps. But unlike Espresso, it is better suited for Black-box testing.

Some of the advantages of using UIAutomator framework:

  • First, scripts created can be executed on many different Android platforms.
  • Second, it’s library is supported and maintained by Google community.
  • Third, UIAutomator allows test cases to run on real Android devices, and its light-weight API is easy to adopt and use for the native apps.
  • Fourth, UIAutomator can lock and unlock a tablet or a smartphone.
  • Finally, UIAutomator can be integrated with testing framework TestNG, in which case, it can generate detailed efficient reports.

Some of the cons include restricted language support with test scripting only in Java and lack of internal view access.

UIAutomation – This is Apple’s JavaScript library that can be used to perform automated software testing on real devices and on iOS simulator. It is suitable for Black box testing.

Some of the advantages of using the UIAutomation framework:

  • First, it offers a simplified UI testing procedure.
  • Second, it offers good support for gesture and rotation.
  • Finally, tests can be run both on physical devices and simulator.

Some of the cons are that it is not an open source tool, hence there isn’t substantial and readily available support. Also, integration with other tools is an area of concern. Besides, the tests are written in JavaScript while iOS code is written in Objective-C and Swift.

As indicated by Indiumsoft, the use of UIAutomation is on the decline as Apple has released a new library called XCUITest as the replacement of UI Automation instruments. Hence, the use of XCTest/XCUITest is expected to rise but the trend is still not clear, and which framework, among a long list, will be able to offer more stable environment is still an enigma.

XCTest – XCTest is Apple’s iOS app testing framework. It enables the users to write basic unit, performance and UI tests for iOS apps. XCTest allows users to do Unit Testing for Xcode projects. XCTest is fully compatible with both Objective-C and Swift, hence, writing tests is an easy task for developers.

Some advantages of using XCTest testing framework:

  • First, testing is simple to initiate and convenient to work, with no need for additional components. It is used with native iOS languages.
  • Second, with XCode Test Recorder in place, testers can get the job done faster with minimal understanding of the underlying software.
  • Third, integrating XCTest with CI is not difficult.
  • Finally, it is much faster than many external frameworks.

Some of the cons to consider are lack of cross platform support, limited ability to test for user condition scenarios — app-context only, a little flakiness on the CI server, limited programming language support, and quite slow to run.

These tools, in addition to many other tools available in the market, assist in accomplishing both speed and quality. Considering the popularity and usage, it is advisable that every tester should know these automation testing tools to remain competitive in the market. Happy Testing!!

Microservices: The Trending Software Architecture – Part II

In our previous blog, Microservices: An Introduction, we provided a broad overview of the architectural composition of Microservices and talked about some of the benefits and challenges associated with it. In this conclusive part of the blog series, we will now discuss some of the key challenges associated with Microservices testing, and strategies to overcome these challenges.

Part 2 – Microservices: Software Testing Challenges and Strategies

The decentralized and independent nature of the development of Microservices poses a number of software testing challenges. Integration of services, preferably over lightweight protocols developed by varied teams in different programming languages using different technologies and frameworks, definitely makes testing a daunting task.

To begin with, Microservices normally display a similar kind of layered internal structure. The testing challenge is to offer test coverage to each layer, and in between the layers of the service, while remaining lightweight at the same time.

Secondly, with multiple teams and technologies involved, the timeframe for availability of each service for testing purposes cannot be specified. Hence, using Web API testing tools built around SOA architecture may not turn out to be a feasible option.

Next, comes the Integration part. For Integration tests, it is essential for QA testers to comprehend the individual services involved in order to create robust test cases. But as data storage is generally managed by individual units, the extraction of logs during testing and data verification becomes a complex task.

Interoperability is yet another challenge that requires Microservices to flawlessly communicate with each other, given the non-integrated independent service development scenario. And further, availability of a dedicated test environment is yet another challenge.

Additionally, understanding and clearly delineating the right type of individual unit testing, and the holistic software testing needed at each stage of the SDLC is not a straightforward task.

Finally, since we are dealing with multiple independent units dispersed in the multi-cloud environment and yet functioning as one, security is a big concern. Authentication is tricky, more ports are open and APIs are exposed making the app vulnerable to attacks.

Let us observe the testing strategies and types that can be employed for overcoming the above testing challenges. A need to reinvent the traditional test techniques along with a thorough architectural and design understanding is must for QA testers. A good approach could be to refer back to Mike Cohn’s Testing Pyramid. Taking a bottom-up approach to testing and delineating the automation efforts needed at each stage can be helpful.

Focusing mainly on automated software testing, five layers of tests can be performed on Microservices.

Unit Testing: This involves testing the smallest part of the testable software in the application to determine if it behaves as expected, hence the scope is internal to the service. These tests are the largest in terms of volume and should be automated, where possible, providing speed and requisite coverage at a granular level.

This testing further necessitates that the other types of tests must also be performed to ensure that the interactions these software pieces have, with each other and with external dependencies, work correctly.

Component Testing: A component here refers to a coherent and independently replaceable part of a larger system. Component Testing involves testing of the overall functionality of a Microservice, in isolation in a controlled testing environment, by replacing the external collaborators using test doubles and internal API endpoints.

This helps to test error cases in a repeatable fashion, in addition to reducing the build complexity, and provides assistance in better monitoring and debugging by exposing internal controls.

Integration Testing: These tests are conducted by integrating individual Microservices before they are deployed to verify that they collaborate as intended to offer the expected behavior. Basic interaction with data stores and external components is ensured and faster feedback is received.

These tests, along with the Unit tests, guide us if the underlying logic is correct, however, they are not enough to ascertain the overall expected outcomes to satisfy the final intended business requirements. Hence, further testing is needed.

Contract Testing: These tests treat each service as a black box, and are integral to Microservices testing. They test the agreed contract for APIs and other resources offered by the Microservice. All services are summoned independently to verify their responses.

Since consumers define the way in which they would consume the service via consumer driven contracts (a specific output against a given input), the aim of contract testing is to ensure that the Microservices meet the contract expectations, even when changes are made to the services.

End to End Testing: This type of testing includes testing the entire system from end to end, to ascertain that the process flows including services and database integrations operate correctly fulfilling the expected business requirements.

These tests can be difficult to write and maintain, have long execution time, and locating a point of failure at this juncture can be quite expensive to fix. Hence, they should be limited in number, and testing at the core levels must be conducted meticulously and judiciously. This level primarily verifies that the entire system meets the external requirements as a whole, and meets the business goals.

All these tests, when complemented with some level of complex manual QA testing or exploratory testing, can help deliver a quality end product– making the service reliable for the end user.

In a nutshell, testing in Microservices can be quite challenging and complex than the traditional monolithic architecture. It’s important to understand the layers and combine different test approaches for a superior end product. Happy Testing!

Microservices: The Trending Software Architecture – Part I

Part 1- Microservices: An Introduction

Microservices, though not a new concept, has recently attracted a lot of attention. Uber, Netflix, Google, and Amazon, among others, have adopted this as an architecture of choice.

Let us delve deeper into the architectural composition of Microservices to understand what exactly it is, and understand the benefits and challenges associated with it.

Microservices, as the name corroborates, is an architectural design that divides a complex app into more logical, isolated, decoupled and independent smaller services. These services are autonomous of each other, and hence, independently deployable and testable, and easy to replace. They communicate with each other using simple APIs. The code for each service can be written in a different programming language, and different data storage technologies can be used, calling for minimal centralized management.

This architecture assists in creating more resilient, scalable and flexible systems that enable rapid development. The necessity to understand the entire code is purged due to relatively short, manageable and easy to read code for each service.

The flip side, however, is that as the numbers of these mini services increase, it becomes difficult to integrate them. Also, they must be scrupulously tested before integration, or later, the implications of a single mistake could be huge in terms of both cost and time.

But how is this architecture different from the Service Oriented Architecture (SOA)?

Microservices Vs SOA

Let’s observe some of the major differences in the architectural styles through the following illustrations:

Microservices Vs SOA - major differences in the architectural styles
Microservices Vs SOA – Source: DZone

1. Microservices are significantly small services that can operate independently and can be autonomously developed, deployed or replaced, unlike SOA. A SOA can be either a monolith or it can comprise of multiple Microservices.

2. Microservices architecture revolves around reactive programming style while SOA centers on imperative programming.

3. Microservices use quick messaging mechanisms, while SOA normally has dependent ESBs (Enterprise service buses).

4. In Microservices, each service can have an independent data storage, while in SOA, services share the data storage.

Now that we understand how the architectures differ, let us focus on the benefits that Microservice Architecture offers.

Benefits of Implementing Microservice Architecture

1. Organized around Business Capabilities: Rather than technical capabilities, Microservices are organized around business capabilities.

2. Autonomous Components: The architecture enables loosely coupled services that can be independently developed, scaled and replaced.

3. Technology Stack Flexibility: For a specific service, software teams using Microservices architecture can introduce a new stack.

4. Agile, Scrum and CI: The software development principles of Agile, Scrum and CI go well with Microservices autonomous components feature enabling faster delivery and speed to market.

5. Focused Functionality: Typically, a Microservice is organized around a single focused capability, and is hence better manageable. The code is small and is thus easy to maintain.

6. Application Scalability: As one application is composed of multiple Microservices that share no external dependencies, scaling a particular Microservice instance is easy.

7. Allowance for Different Languages: Different services can be written in different languages for later integration into the main application, thus embracing flexibility and technology diversity.

8. Decentralized Data Management: Decentralization allows for teams to work independently as each service manages its own database.

9. Automated Deployment: With independent implementation, Microservices can easily be moved from one deployment environment to another.

Challenges Posed by Microservice Architecture

1. Compatibility Issues: Built by different teams using different technologies, Microservices often face compatibility and integration issues. This results in an unstable environment.

2. Increased IT Costs: Implementing Microservices with Big Data Project can be quite costly. Hiring skilled experts and training staff to learn new codes for integration and getting access to additional servers and database licenses can prove expensive.

3. Application Security Issues: With different OS, frameworks, and languages, it is not easy to keep the system secure from hackers. Also, there are not many Open Source tools, frameworks, and models to support Microservice dependent software.

4. Complex Fault Tolerance: In this architecture, service failure can go multifold creating a cascading effect and is difficult to handle than in a monolithic system, especially with a high number of services and processes.

5. Resource Rivalry: With the development of a few dozen Microservices in the process, it becomes difficult to ensure access to scarce, complicated and expensive hardware and engineering resources to all the teams.

6. Monitoring Troubles: System state keep changing in Microservice architecture, and it becomes essential to remain aware of the system at all times. A lack of this may result in system failure.

7. Stability Issues: Frequent changes and faster deployments with different technologies and languages coming together, it is quite difficult to ensure stability.

8. Testing Complexities: Managing the testing complexity of multiple independently deployable components is a daunting task for testers.

Conclusion

Adoption of Microservices Architecture is currently a topic of major interest in the app development community. The architecture offers faster delivery, greater operational resilience and scalability, decentralized data management, and technology diversity. However, like other architectures, it also has its own set of challenges. Higher costs, resource scarcity, monitoring troubles, testing complexities, stability and compatibility issues, all pose failure risks.

Despite these risks, the adoption of Microservices can be highly effective if the right tools and solutions are used to address operational and security issues. In our next blog – Part II- Microservices:  Software Testing Challenges and Strategies, we will be focusing on the specific software QA testing challenges that Microservices pose, and the testing strategies that can overcome these challenges leading to a secure and reliable system.

7 Critical Software Testing Skills

Most software engineers join the workforce right after graduation. Historically, software test engineers have found themselves at the bottom of the pyramid– arguably so. Manual testers, especially, fail to acquire significant hardcore programming skills early in their careers. They normally do not get considerable latitude to learn these skills. At this point, and onward, many tend to become complacent with time and end up sticking to QA & testing as a career choice. For most QA professionals, their learning curve starts dwindling, especially with new and continuous technological advancements. The current scenario, however, does not allow this at all and offers ample opportunity for those willing to step up their game.

Agile testing is the new buzzword, as faster deployment is becoming the key to stay ahead of the curve. Automation, lean processes, and rapid turnarounds with the minimal possible post release defect percentage are crucial for success in the current competitive environment. With many companies developing similar products and offering similar services, quality has become a major differentiator. The role of the tester has transformed dramatically with the need to work with and among developers, designers, product managers, customers, and clients.

Software Testing has evolved from ‘no need for’ technical know-how to ‘need for’ an in depth knowledge of software testing development life-cycle, out-of-box thinking, and a fair amount of programming skills. Hence, continuously learning new things to keep up with the latest trends has become inevitable.

Here are seven critical software testing skills that a tester should have or should learn to remain competent and relevant in the contemporary dynamic landscape.

1. Networking and Communication

Whether connecting with experts, exchanging ideas, or sharing subject related pertinent information, social networking help testers to remain up-to-date with the latest testing trends and techniques. And communication skills especially soft skills play a vital role in this.

Additionally, communicating effectively with professionals, with stakeholders, and with team members assists in better comprehension of the business requirements and testing needs, resulting in a robust test plan.

Some other significant deliverables of good communication skills are – structured test cases, apt bug reporting, and clear and concise management reports forming a solid basis for a viable end product. Further, these skills also help in adopting the latest approaches, like DevOps, successfully.

2. Agile and DevOps Methodologies

Testers must be adept at Agile and DevOps methodologies for continuous releases and collaborative teamwork. These approaches form the basis for software development in the current dynamic environments, eliminating role rigidity and silos.

3. Automation

Manual QA testing cannot entirely cope up with the increasingly complex frameworks and testing scenarios. Hence, automation testing skills are must for higher accuracy and faster delivery. Testers need sound knowledge to figure out the tests that need to be automated, the tools required for the purpose, and the execution methodology. They should also be well aware of various bug tracking tools, GUI testing tools and more.

Some of the sought-after tools testers should be aware of include– Selenium, including Selenium Remote Control, QTP, LoadRunner, xUnit frameworks, JMeter, Soap UI, Appium, and Calabash among others.

4. Latest Web and Mobile Technologies

To deliver better QA solutions, testers must be aware of the web and mobile technology advancements. It assists them to better understand the coding architecture, frameworks needed, and the technical aspects. This will help them to better understand the app built and scalability issues for testing purposes.

Some technologies that are frequently used include XML, HTML, Javascript, and CSS. Similarly, testers must be aware of the protocols like HTTP and HTTPS.

5. Testing Techniques and Basic Programming Skills

Every tester must be aware of the various testing techniques and tool usage. The knowledge of black box testing, penetration testing, unit testing etc. help testers work effectively on the different type of projects.

Learning some core programming skills can further assist testers to work in a variety of environments. Python, Java, Ruby, and SQL are some of the popular and in-demand programming languages. If testers are aware of the fundamentals of these languages, it can help them to understand the basic app functioning, to figure out the coding errors, and hence, to create effective tests.

6. Understanding Priorities

Time management skills are crucial for testers. Understanding what should be tested first and foremost, and what can be left for later is critical. Similarly, a tester must be able to figure out what can be automated and what needs to be tested manually in the given timeframe and as per the need of the project. With continuous releases, delineating prioritized features and testing them aptly for a defect-free timely release is critical.

7.  Logical Thinking, Curiosity, and Problem Solving Skills

A tester with a curious and analytical mind is able to better understand the changing business requirements and associated testing needs. These skills are crucial to get the requisite clarity on the project by continuously communicating with the different stakeholders.

Additionally, an out of box thinker, with a logical and analytical bent of mind, is more capable of finding bugs, understanding the complexities of the app, choosing the right tools and techniques, comprehending different testing environments and frameworks, and accordingly testing the app. Experience, no doubt, also plays a major role in this.

Similarly, with problem-solving skills, a tester is able to question unusual app behavior, analyze it well, identify defects and defect probabilities and look for possible solutions, or prepare an effective bug report to deliver a robust end product.

Conclusion

These are some of the software testing skills a tester must possess to ensure that the project is running smoothly. Software delays are costly for a company and so are defective releases. Hence, a tester must keep up with the latest technology, and continuously learn and update testing skills and knowledge to remain competitive and relevant.

So, keep learning and improving yourselves testers, you are doing a great job!!

 

Exploratory Testing and Agility

The continued shift from the rigid Waterfall Model to the dynamic and fast-paced Agile approach reflects the way the software development industry is changing. And with highly competitive enterprising business environments becoming the reality, Agile is the way forward.

The Agile approach definitely has its own set of merits. It endorses adaptive growth, frequent releases, higher stakeholder involvement, reduced documentation, better collaboration and increased emphasis on automation. And Agile works well with Exploratory Testing. Though exploratory testing does not have a predefined sequence of steps; it is flexible and allows testers to adapt their testing strategy depending on the findings of the test, thus making the process faster.

Source: Testing Geek

Let’s observe how exploratory is compatible to the Agile projects.

The Importance of Exploratory Testing In Agile

1. Agile development quickly adapts and adopts the changing business requirements, and exploratory testing validates these requirements in an effective and pragmatic manner. With Exploratory Testing, there is no need to invest time in authoring, preparing and approving test cases before the testing begins, and hence, the test process is fast and quick, and in sync with the agile methodology. This can help in testing significant scenarios in a limited timeline.

2. Exploratory promotes collaboration – a tenet of Agility. Without any basic requirement to gain test automation or scripting knowledge, most team members including UX designers to developers can collectively contribute to the testing efforts thus leading to a better end product.

Moreover, exploratory testers in an Agile environment get a better opportunity to interact with all the stakeholders enabling better understanding of the product and hence better test cases for efficient solutions.

3. Whenever any part of the software is released iteratively each month, both automated and manual testing is normally conducted. Exploratory testing is helpful in finding defects especially when automated software testing is not a viable option. Whenever a new feature is added to the app, the tester is required to understand and create test cases and then automate them, which is a time-consuming process. Agility, however, demands quick results, and hence, a manual adaptable approach is needed. Here, exploratory testing helps in catching the bugs quickly, and well in time. Only when the test is important enough to script and execute repeatedly, automation should be considered.

Besides, all automation is anyway not sufficient enough to ensure that the system works the way we expect, and the system has to be used by someone to ensure that, hence we rely on exploratory testing. Exploratory testing cannot, however, be a substitute for automated software testing, it rather complements it well.

4. Lack of planning and documentation may leave a few undetected bugs and risks. By means of exploratory testing, a tester properly studies and analyses the software product, identifies and detects associated risks, develops the test cases and executes them, thus leading to a quality end product. The creative thinking and quick response time are hallmarks of exploratory testing. Additionally, leveraging human intelligence and wisdom, exploratory testing offers a broader and deeper view than any automated test can.

5. In the Agile approach, exploratory testing efficiency advantage can be a blessing reducing the budget overheads and complexity of QA automation. Formal test documentation approach, on the contrary, can be prohibitive especially when the budget is tight and resources are scarce.

6. Exploratory testing uses the results from the last test to inform the next, so the test approach can be adapted for vulnerabilities or risks observed/noticed. The information can assist in focusing on the current scenario than following the test scripts written in the past. This shows its compatibility with the Agile testing approach.

7. Exploratory testing is extremely useful in complex testing scenarios where the test specifications/requirements are not clear or where the domain knowledge is lacking, or when there is little time to specify, script and test. For untested, unknown and unstable products, exploratory testing is a savior. It’s called in anytime the next test to be performed is not clear, or when testers need to go beyond the obvious, which is a common scenario.

Conclusion

In a nutshell, exploratory testing cannot be termed as an ad-hoc testing approach rather it refers to a thoughtful and creative testing approach tied aptly to the Agile methodologies. This results in creating an excellent synergy within the testing sphere, thus resulting in a superior quality end product in the minimum possible time.

The Art of Producing A Quality Bug Report

Whether we speak of manual or automated software testing, the goal of software QA is to minimize bugs. Bugs are an integral part of a software code, and in order to fix these bugs early on, they need to be identified and communicated clearly and well in time to the development teams. This is done via bug reports. Fixing these bugs, however, largely depends on how effectively these bugs are reported.

Bug writing is a skill, and a well-written bug report is objective, concise, clear and contextual. The significant details are not lost along the way, nor is a good report biased by the judgment of the person writing it. Bug reports serve as the main communication link between testers, developers and project management teams, and must be written with a perspective to offer each bug the best chance to get fixed.

Before we understand how to effectively fill different fields in a bug report, there are heuristics to abide by:

  • Assign a unique number to each bug report for easy identification of the bug record.
  • Keep a track of the exact steps to reproduce the bug.
  • Be precise, concise and contextual when reporting the bug.
  • Produce different reports for different bugs.

These steps lay down the foundation for a quality bug report document.

Significant Bug Report Fields And How to Effectively Fill Them

Title: Short, simple and precise, a good title should provide enough context to let the reader understand what the bug is all about.

Environment: Bugs tend to appear in specific environments, hence it is advisable to mention the OS and browser details along with hardware and software versions being used.

Severity: Priority of bugs in a backlog is an important piece of information but this should be considered in the context of the end user besides the business’s decision to mark its priority relative to other work.

Expected Behavior: To get a clear understanding of the actual bug, it should be written in a way that the person reading the bug report is able to understand the context and the desired/expected behavior of the program.

Actual Behavior: This is the crux- the real matter. So, take the liberty to provide details, and if there are a number of things that are not working as expected, consider creating multiple bugs (or maybe a parent bug with sub-bugs).

Steps to Reproduce the Bug: Clearly specifying the step by step process to reproduce the bug usually makes things clear- what was expected, what actually happened, and the environment used. If a bug is not reproducible, it’s either a user error and if not, the likelihood of getting it fixed is quite low.

In addition to effectively filling up the above fields, it is equally important to proofread your bug report before submission. Incorrect or confusing language may cause unnecessary delays and lower chances of bugs getting fixed. Also, avoid opinions, use constructive language and be precise and factual. Besides, make sure that the bug has not been reported earlier- duplicity can create chaos. Finally, the bug report should provide sufficient data for an informed business decision, which may even be to not to fix the bug at all.

Need to Report the Bugs Formally

No doubt, testers are in the business of reporting bugs, but why to keep a record of each and every bug? To say the least, it’s better to be proactive and reach out to developers directly when required and to keep a record of all the bugs because this will ultimately lead to a best possible product outcome. Even bugs that are unrelated to what testers should be concerned with offers customers a chance to evaluate their pros and cons if reported.

Hence, even if this isn’t the time to fix an error or if the bug requires further investigation, testers should simply post a sticky note to the board within the sprint to keep the team aware. Regular conversation with Product Owner can also help clarify whether the bugs found need to be resolved or are insignificant. Reporting a bug formally is, therefore, a good practice leading to a better end product.

Conclusion

Writing a high-quality bug report is a skill, and hence, it’s important to spend appropriate time on this task to offer a concise, correct, complete, courteous and constructive bug report. The report fields should be filled in effectively and should contain enough information to enable an informed business decision.

However, even when a bug report is written cautiously, there are factors that can reduce the effectiveness of the report like communication barriers across time zones, cultural and language issues, inexperienced testers and pressure to deliver the product fast. These issues can be resolved through proper management and training, and can significantly impact the bug reporting mechanism and ultimately the end product when left unsettled.

So, take care of all the above and you are on the journey leading to a high-quality end product. Happy bug hunting and reporting!!

Augmenting ROI en route QA Automation

In addition to higher efficiency and accuracy of the testing process, QA automation has often been associated with increased ROI via better speed, superior quality, reduced cost and optimal utilization of resources. However, automation cannot be applied indiscriminately. Being a strategic decision, it requires a fair amount of understanding as to what it can or cannot offer to your project. Project teams and QA decision makers must evaluate the potential ROI before making a final commitment to invest in software test automation.

Once the groundwork is done, and you have set your foot on the automation playground, the initial step is to get acquainted with the factors that can help you reap the desirable ROI from the process.

Identify What to Automate: To identify this, you need to study and analyze factors like the application under test, requisite test environment, hardware dependencies and so on. Tasks that are recurring and mandatory in nature, or functions that remain stable for the most part of the development lifecycle, or tests that have complex business logic constitute some of the ideal test cases for mechanization.

Moreover, the important question is whether, in an attempt to accrue the benefits of automation, can we automate each and every layer of the app and its related components, and be done away with the human element? The answer is a crisp ‘No’. Why?

Here are a few good reasons :

  • Even Automation Test scripts need to be written by testers/humans
  • Scripts cannot provide inputs on how a UI looks and feels
  • Automated tests can only notice the errors they are programmed to find
  • Automation can prove to be a costly affair for smaller projects
  • Only skilled, wise testers can choose apt automation tools or the effort can be a sheer waste

Hence, both manual and automated testing are crucial for a successful app to be developed in a given time and budget, but yes, automation can increase the benefits manifold when applied aptly. Hence, choose wisely what you really need to automate.

Task Prioritization: Once the test cases for automation have been identified, it is imperative to start prioritizing and dividing tasks as per the expected timelines. It is equally important to consider team skill-set and accordingly finalize the automation tasks.

Prepare Test Cases & Identify Scenarios Ahead: This step assists in identifying the estimated execution time and possible defects well in advance, in addition to identifying other scenarios and application workflows that can also be automated but could have been otherwise missed.

Identify the Test Process Beforehand, and Create Tests That Do Not Affect UI Changes: Identifying Test Process before going for automation ensures optimal allocation of resources and early risk analysis. This assists in the determination of best possible methods to perform the task efficiently. The automation scripts must, however,  be easy to maintain and update. Finally, tests should be so written as to provide smooth and reliable results even when the UI changes.

Ensure Well-designed Test Data: Test data that is fresh/updated, valid, and reliable results in successful automation endeavors. Hence, make sure that you have quality data – optimal size data with information and components relevant to app testing environment.

Identify Apt Test Tools, Frameworks and Techniques: To obtain the desired output from the test automation project, you need to choose the right test tool- emulators or real devices, make the buying decision- buy the device or use the Cloud services, and select the right automation framework or technique for a particular app type or test scenario.

Periodic Test Review: Periodic review of your test data and cases is essential for eliminating unnecessary and outdated tests resulting in reduced costs and maintenance efforts.

Hence, automation, when implemented aptly, changes the entire test environment from design to deployment for good. Speedy and efficient delivery, better visibility into your app performance, detection of more bugs, and cost effectiveness are some of the hallmarks of automation that can be expected when you tread the automation journey in sync with the above guidelines. So automate, but automate wisely!

Service Virtualization In Software Testing

The complexity of the current QA software testing market impels businesses to strive for quality software quickly and economically. DevOps and Agile workflows have made the software testing ride much smoother than the traditional practices by introducing automation and enabling superior communication, collaboration, and transparency. However, waiting on the dependent components can stymie even the best of approaches, and this is where Service Virtualization (SV) can help speed up things.

What Is Service Virtualization

In the absence of key components in your system architecture, SV involves using virtual services to enable frequent and comprehensive testing by emulating the behavior of essential components. This means testing teams get a comprehensive first-hand testing platform equipped with all the components of a real production environment, enabling testers to test component-driven applications such as independent APIs, SaaS-based apps and SOAs (Service-oriented architectures).

Service Virtualization In Relation To Stubbing and Mocking

Modern applications are complex and rely on numerous dependent services. Adding to the complexity of software functionality, the rise of Agile software development has made it increasingly difficult for testers to manually develop the number, scope, and complexity of stubs or mocks required to complete testing tasks for modern enterprise application development scenarios.

Service Virtualization should not be mixed-up with unit testing with stubbing and mocking– which are mere workarounds, unlike properly architected SV technology. With stubs and mocks, the test suite simply ignores the unavailable components, often leaving the vital components out of the testing sphere until a final end-to-end testing is conducted prior to the release. The major advantage with SV is that testing teams have the ability to virtually test application behavior incrementally before the full availability of all components. This largely eliminates some of the major disadvantages of stubs and mocks, making SV a valuable asset for the testing companies.

Advantages of Service Virtualization in Software Testing

Let us highlight some of the key benefits Service Virtualization offers:

Speedy Delivery: In the current Continuous Development (CD) scenario, testing needs to occur alongside development, and this is especially desirable in the production of heterogeneous systems involving multiple layers of interdependent components, APIs, and third party apps. It is no longer feasible to wait for QA teams to give a green signal for each and every component to be market ready; rather the behavior of the connected components can be understood in a demo environment using SV. This leads to reduced timeframes and shorter release cycles. This is further validated by the voke Research 2015 where 34% participants experienced a 50% or greater reduction in test cycle times, while 40% participants saw their software release cycles decrease by 40% or more using SV.

Access to Otherwise Unavailable Resources: A complete end-to-end test can be conducted even when the dependent system components (third party apps, APIs and so on) of the app under test cannot be properly accessed or configured for testing. SV helps to simulate these dependencies. Moreover, almost all kind of scenarios can be tested using SV, including varying levels of functionality, performance and maintenance levels.

Reduced Costs: Operational costs can be reduced significantly through a planned and systematic approach reducing test environment configuration time, easy test environment access and setup, and elimination of interface dependencies. Moreover, since each component can be tested individually without waiting for the complete assembly, unit and regression testing can take place sooner, is more complete, and bugs and performance issues can be identified long before integration or user acceptance testing, making resolution possible early in the SDLC, thus saving huge remediation costs. Infrastructure and resource costs are also significantly saved. This can further be justified in the light of HPE Service Virtualization Case Study that depicts cost savings of £1.94 million through SV.

Reduced Business Risks and Increased ROI: With the ability to test early and often, defects get exposed when they are easiest and least costly to fix. Early detection of bugs implies reduced risk of defects slipping into the final product and faster delivery, ensuring that businesses stay on top of their competition in a cost effective way. This reduces the business risk of product failure and offers a superior ROI through speedy product delivery. HPE Service Virtualization Case Study depicts an outstanding value for money using SV,  yielding an ROI of 88.6%.

Better Quality: The actual product deployment scenarios can be mimicked with SV making it easier for QA teams to identify issues and failures before the product goes LIVE for users. Development errors are caught well in time through the shift-left approach with enhanced scalability, ascertaining a robust end product. As per the voke Survey 2015, 36% participants reported a reduction in production defects by more than 41% by adopting SV, while 46% participants experienced more than 41% reduction in total defects, thus resulting in a superior quality product.

Service Virtualization, hence, reduces the time, effort and cost of delivering secure, reliable and compliant software by eliminating numerous software testing constraints. It is, therefore, a smart investment for quality assurance software companies, culminating in measurable, tangible benefits. Happy testing!!

Agile Scrumban – Built on the Fabric of Scrum and Kanban

Scrumban, a hybrid of Scrum and Kanban flavors of Agile, offers the best of both worlds. Scrumban ensures that a highly versatile workflow management is achieved by integrating the flexibility and visualization of Kanban into the structure of Scrum. Scrumban thus offers a middle ground to the Scrum’s structure overhaul, and Kanban’s unstructured approach – making it useful for both development and maintenance projects.

To fully understand Scrumban, let us first understand the main individual characteristics of Scrum and Kanban.

Scrum

  • Small, cross-functional self-organizing teams
  • Work is split into a list of small deliverables, sorted by priority
  • Prescribed roles- Scrum Master, Product Owner, Development Team
  • Short fixed length iterations with potentially shippable code demonstrated after each iteration
  • Approach is to inspect and optimize the release plan, and update priorities
  • Retrospectives occur after each iteration

In Scrum, the work to be done for the next sprint is chosen beforehand, and the sprint is then locked till the work is done. And at the end of the sprint duration, the queue is empty.

Kanban

  • Workflow Visualization through Kanban Board
  • No prescribed roles
  • Limits WIP – limits the items in process at any given time
  • Measures lead time and optimize the process to improve the lead time

In Kanban, the work keeps flowing, and although the size of the queue is limited, the items in the queue can be changed at any time.

Scrum and Kanban: The Divide

Both these approaches have their own pros and cons and work on different ideologies. Whereas Scrum has a set of prescribed roles, Kanban has none. Meetings are an integral part of Scrum, while Kanban has no such requirements. Besides, unlike Scrum, Kanban does not have a fixed time limit for development.

Let us observe how Scrumban fixes this divide and coalesces an ingenious approach absorbing the best of both.

Scrumban

Introduced by Corey Ladas, Scrumban was initially developed as a method of transitioning a development team from Scrum to Kanban. However, it has evolved as a methodology on its own, combining the finest elements of both Scrum and Kanban.

Scrumban Methodology: Filling the Void

  • Scrumban makes iterations short and optional while imbibing Kanban’s approach of a continuous workflow. Items are worked upon as and when they appear, with no prioritized and committed product backlog items. Prioritization on demand technique is normally used.
  • Scrumban has a definite team, but only requires roles as needed. There is no definite number, and roles are more specialized and less cross-functional than in Scrum teams. Team members are free to choose the task they like using the pull system.
  • Scrumban boards are used like Kanban for workflow visibility with the columns – ToDo, Doing, and Done. The board is persistent – only the tasks and priorities change.
  • Like Scrum, Scrumban involves daily meetings, however, embraces on-demand planning at regular intervals, rather than sticking to release-planning meeting or retrospectives at the end of the sprint like Scrum.
  • Like Kanban, Scrumban does not have a specific time constraint. Through the use of WIP limits on columns within their task board, the team limits itself. In Scrumban, a WIP limit is normally equal to the number of people in the team but can be expanded based on work specifications.
  • Like Kanban, Scrumban relies on measuring the lead time and cycle time as the key metric to estimate the average time of completion of a specific task.
  • Feature Freeze is used in Scrumban when the project deadline is near, which implies that the features the team already has for development can only be worked on, and no additional features can be added.
  • Finally, triage is critical as Scrumban does not embrace estimating. Scrumban triage enables termination of less significant features in order to complete essential features on time.

Scrumban definitely brings in the best of both worlds, however, application of Scrumban is not conducive to every environment and culture. Scrumban is apt for fast-paced processes or projects that require continuous product manufacturing with dynamic environments. Let us understand the applicability criterion for Scrumban.

Suitability of Scrumban

To sum up, Scrumban is ideally suited for:

  • Projects that involve a great deal of unexpected change to user stories and priority reworking. For example- maintenance projects like event driven work or help-desk/support.
  • Teams focused on new product development or continuous management improvement.
  • Projects where Scrum is constrained by workflow issues, resources, and processes.
  • Teams that require both structure of Scrum and flexibility of Kanban.
  • Transitioning to Kanban, where minor methodology changes are required to limit disruption.

Indubitably, when used aptly, Scrumban leads to better quality, reduced lead time, application of Kaizen and JIT principles along with waste minimization and process improvement. A word of caution is that before applying any methodology, it is imperative to understand the basic principles of all the agile methodologies currently in practice to determine the befitting approach for the project specific needs. The company and team readiness to adopt that approach also matter.

With skilled resources and expertise to understand the intricacies of various agile approaches, we at Astegic have developed a Scrumban Framework – SBDEFT (Scrumban Driven Engagement Framework for Testing). SBDEFT is developed with an aim to deliver the benefits of both the approaches, Scrum and Kanban, in a pioneering way in an outsourced context. Our experts are adept at understanding and implementing the approach that best suits your project, whether it is Scrum, Kanban or Scrumban. Moreover, our TRAF (Testing Requirement Analysis Framework) ensures validating QA /Testing requirements in the early stages of development to serve you with the best approach.