As the software development industry moves forward, one thing that never stops is the ever-evolving introduction of new concepts and/or buzzwords. Some are literally just buzzwords wrapped around something that previously existed in the industry in some form or the other. However other concepts go beyond just buzzwords, and need to be taken seriously as it potentially becomes a key factor in influencing how software is and will be built going forward.
Failing to keep up with the concept or trend could mean that previous tried and tested approaches might not be as effective and even worse, might put major question marks against an individual’s relevance and marketability in a thriving, fast paced industry. One such concept is that of “Microservices” or a “Microservice Architecture.” With this concept/approach in mind even more questions come to mind regarding Development, DevOps and other areas, however what does it mean for previous quality engineering, testing and test automation approaches?
If the ground under you moves while standing, naturally it would mean you change direction, or initiate a different response to counter the movement. The same goes for a moving software architecture, your quality engineering approach has to adapt to this gradual or sudden move. In this post I will share some points to keep in mind when tackling your test automation approach in a microservices architecture environment.
What are Microservices in a Practical Setting?
Before we tackle the test automation approach in a microservices architecture, let’s take a quick look at what microservices actually are and a simple practical example of something that you use often that might simplify your understanding of the concept.
A microservice architecture is usually characterised by focused, small, independent services that together create a complete application. Every instance of a particular microservice usually represents a single responsibility within your application.
A practical example would be depicted on something like an online banking application that most of us use on a regular basis (the same concept is applicable to any industry/type of system). In the past, typically a traditional architecture would have all functionality of a banking application bundled as one large system, most often with code referencing different areas of the application in a single deployable package. While these systems still worked, they were often plagued with many shortcomings like:
- Maintenance nightmares: Amending one feature would impact many unrelated areas due to code dependencies. Pinpointing and digging in to find where a change should be made, in itself took quite a long time.
- Performance issues: Consider a busy highway, in peak hour traffic with no alternative routes or off-ramps to take. Each car wants to get to a slightly different destination but has to pass the single, busy highway to get there. What results in that scenario is a bottleneck of slow-moving cars that cannot get to its intended destination at a given time. Traditional systems were marred by a similar dilemma. A single path to get to perform either the same or slightly different actions or transactions (for example 1 user wants to make a payment and the other wants to open a new account would follow specific paths in a single code base).
- Lack of control or flexibility: If a new feature (like international payments) needed to be deployed to production often it meant that the entire application or system needed to be taken down to deploy the change. This often meant a user who wanted to perform an unrelated action like to check his/her balance or even just try to change their username now had to be inconvenienced by the entire application being taken down and they would be prevented from performing their desired action until the site/application came back up again.
There are many other shortcomings of traditional ‘monolithic’ systems, the above just highlighted a few.
Enter microservices–to ease some of the above shortcomings and to cater to a fast-paced, high demand world, microservices brought in a fresh approach for tackling software delivery.
What a high level microservice architecture would look like in the above online banking application example would be a split of key services/features or even business areas to optimise how software is built, tested, deployed and maintained. So in our example now there would be a split of your key functional areas like a separate “Payment” service, that handles all payment related functionality, then possibly an “Account” service which contains all information about accounts, balances etc. Further to this you could also have a “Customer” service, a “Reporting” service and so on.
Most of the services might be able to function in total isolation but often it does need to have a dependent service available to complete a transaction/task. For example, to complete a payment, the payment service would still need to get information from both the customer and account services in order to initiate the transaction, whereas in this flow the reporting service would not be a dependency to complete the transaction.
That’s a lot of context-setting :-). Now let’s move to what this means for your automation testing approach.
What Did Test Automation Approaches Look Like in a Traditional, Monolithic System?
With traditional systems, approaches to test automation needed to take (in my view) a restrictive approach. Restrictive as the access into and around lower areas of an application was very difficult, non-exposure to integration points also often meant that some had to create customised test hooks, harnesses and so on. Due to the complexity of this customisation most people just chose to go for the “What you see is what you get” approach
The following points list out some characteristics or results of what typical test automation approaches looked like:
- Limited automation tools choice: As systems were less flexible, test automation tool options were limited in what they could do and the creativity/innovation of options at hand were quite one dimensional. With the evolution of modern system architecture, it has allowed automation tools vendors and/or individuals to come up with more flexible solutions to cater to test automation needs.
- Long running tests: As mentioned, tests had to be focused mostly on the UI as this was one area of the application that was easily accessible, or if the test hooks were created to access a lower level, the setup was tedious and often resulted in an increase in test execution times. The lack of flexibility, limited access plus the absence of clearly defined boundaries of an application created this dilemma. This also meant that when tests were executed it was a mission to pinpoint errors in the automation script or where in the application an actual defect existed.
- Maintenance nightmare: Similar to the point of a monolithic system drawback in itself, test automation also shared a similar problem. Test automation was seen as a separate, outside activity characterised by siloed test automation repositories, with ownership by a separate test automation team or QA team. Tests had multiple aims/objectives which made the code overlap, long-winded and difficult to maintain.
- Delays between deploys and testing: The very nature of how large, traditional systems were built and deployed meant that there were clear distinctions between Dev and Testing activities. Code deploys followed by a notification to QAs to execute the test scripts meant that the feedback loop now was longer than what we see in teams/setups where microservices architectures are coupled with proper Agile processes.
- Performance/Load testing as part of one big bang approach: Performance/Load testing is usually tackled as one large system E2E effort. Once again due to system limitations and lack of flexibility even Non-Functional testing like Load, Performance and Security testing are forced to be done once all software development is complete, integrated (with its dependent systems) and deployed.
How Should You Now Refine Your Test Automation Approach to Cater to Microservices?
Thankfully modern system architecture solves many of the drawbacks of traditional systems, and the same is true for the opportunities presented by modern test automation approaches which breathe new life into how we tackle test automation.
How does your automation approach now change to fit in with modern architecture?
- Greater test automation tool choice and toolset: With the greater ability to access all areas of an application like APIs, UI components etc., test automation tool options have now grown immensely, which bolsters the ability to focus on specific tests targeting specific areas based on intent.
- Lower level testing: Testing individual components/services in isolation now becomes very realistic and much needed in order to test as one builds. Incorporating aspects like mocking, containerisation, contract tests etc. gives one much more power in how to structure your test automation efforts as software is built.
- In-sprint automation, focused building, focused testing: As the point above depicts, this brings the added possibility of not leaving test automation as an after activity, but rather something that gets incorporated in your sprint efforts, as code is being developed.
- Non-functional testing per service, as you build: While isolated services bring greater flexibility in functional test automation, it also brings added opportunities for focused Performance and Security testing. This gives one added control in pinpointing exact NFTs at individual service level so that optimisations can be made to fix bottlenecks or vulnerabilities. Here you would think about performance testing your APIs, or a specific component of a UI, checking for security vulnerabilities of each API and much more.
- Automated deployments with Automated tests: Even though some form of CI/CD is still possible when dealing with traditional systems, its use case is heightened when dealing with microservices. Test automation tests no longer operate as a separate, siloed entity, but due to its smaller, focused intent, now becomes part of the build, deploy pipelines to ensure one builds in quality from the outset.
- Bring it all together with a pinch of Integration and E2E system level tests for reassurance: Depending on the number of dependent services/applications one has in their given ecosystem, sometimes the need arises to make sure that your tests also provide some extra reassurance that everything works together by incorporating integration tests or system E2E tests focusing on multi-service, multi-system touchpoints. This is where adding in a form of system level test automation makes sense to provide extra confidence in your test automation efforts.
Challenges
While there are many benefits of microservices and a refined test automation approach, there are still some areas which provide huge challenges and pain-points. These include:
- Greater dependencies on multiple services: As services are not used individually in the customer facing world, when they are individually tested one needs to make sure it resembles a real-world use case given its implementation. Usually this would mean a foolproof mocking approach plus making sure that relevant mocks are kept in-sync and/or maintained regularly.
- Closer control around test coverage: As there are now multiple services usually owned by multiple teams, making sure everything is covered becomes a critical balancing act. Measures need to be put in place to address tracking of coverage per service, how to tackle missing areas of coverage and plan of action to handle these risks.
- Skillset gaps: With newer test automation tools, approaches like lower level testing forms including service virtualization etc, most test automation experts or team members would need to up-skill in this area as it requires a specific skill set to perform activities needed here to provide maximum benefit.
- Test data considerations: Availability of test data is usually a huge factor across most contexts, and the same is true for microservices, while in a way there is greater control in creation or mocking of test data in a microservice test automation approach, making sure the test data is representative of real-world scenarios is also quite tricky.
- Possible need for ownership of cross system E2E aspects: If some of the points above have gaps, then a greater need for coordination of cross system E2E test automation areas becomes important. This is to make sure that the sum of the parts make a proper whole.
Conclusion
As the way systems and applications are built evolves, so too does our QA craft and test automation approaches. While every company operates differently, there are usually some overarching areas or guidelines that one can keep in mind when considering how to tackle your test automation. No solution is without its challenges, but one thing for sure is that as we evolve, we are presented with opportunities to constantly improve and make sure that we never forget the aim of why we choose to automate our testing efforts.
