The Agile Manifesto is 16 years old. In the home page, it defines a very simple set of values:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
These four simple values have sparked a revolution in how we write and deliver software. I like to call them the “cut the bulls**t” values. To understand how these values affect testers, let’s dispense with the ceremony, let’s dispense with lengthy design documents, and, yes, lengthy test plan documents, and focus on understanding the core — writing the software.
And the way to do that is to deliver working software from the start (this is the “Working software over documentation” part of the manifesto). The agile manifesto asserts that the product needs to work from the beginning, and not wait for a lengthy and cumbersome “integration phase” when all the work on the product is integrated together. Instead of many team silos, where everybody is working on one piece of the puzzle (and the architect is the only one aware of how everything will be pieced together), we start with a small team that builds an initial working software (MVP, or minimal viable product) and gradually grows it to a bigger product with a bigger team.
This team includes everybody — the customer, product manager, developers, and testers (this are the “Individuals and interactions” and “customer collaboration” values of the agile manifesto). This integrated team delivers a minimal version of the product in a very short time, and… waits. Waits for what? For the customer to start working on the product and figure out the next step for the product. In other words, agile teams don’t build a specification for the whole product, but rather start small and incrementally determine how the product should look like (this is the “Responding to change over following a plan” part).
Yeah, yeah, yeah. Bla, bla, bla. I got that. But how does it affect my work as a software tester? It does. Immensely. I have one sentence for you:
Embrace agility: shift left and shift right.
But, unfortunately, to explain this, and to see how this affects your work as a software tester, I need a bit more explanations, so hang in there.
Testing in the Days of “Release Early, Release Often”
The agile manifesto also says: “Deliver working software frequently”. And today’s software development community has taken it to the extreme. Some companies release every week, some every day, some don’t even have the concept of “release”. They just release anytime a feature is ready.
Moreover, agility forced the backend to come up with an alternative to the “monolithic backend”. The concept of “microservices” has replaced the “monolith”. This concept enables large companies to break up their backend into many small services, each one responsible for a small piece of the puzzle, and to deploy each microservice independent of each other.
These two things means that in many companies there is no “release”. It’s a fluid set of changes that just happen to the product in ongoing process.
But if there is no release, how will software testers test? I have one sentence for you:
Embrace agility: shift left and shift right.
Shifting Left
What does shifting left mean?
The software development software, in the days before the process became agile, looked like this:
Software testers used to be at the right of this process. They used to be the gatekeepers that bar the gate to releases that are not qualified for deployment.
This process actually hasn’t changed much in the days of agile, except that now we have many of these pipelines in parallel, and each takes days, and not months:
But how can a software tester be a gatekeeper in this model? It’s practically impossible. Once the implementation is done, there is no “time” for testing, as the team has gone on to the next thing it needs to do. In the agile model, delivering software incrementally is paramount. Time for testing is not included.
To solve this, the tester needs to shift the testing left. It needs to be part of the implementation, and not a separate step:
This has many practical consequences, consequences that affect you as a software tester:
Software developers need to test too
Shifting left means that a lot of the burden of testing needs to be dealt with by the software developer and not by the tester. A tester should not need to check that a certain form field needs to be numeric. A tester should not need to check that the value in this table is correct. All those boring tests that occupy a lot of the tester’s time need to sit “close to the code”, and should not be dealt with by the tester. The software developer can use unit testing to check those boring checks in much faster and more reliable manner than the tester, and free up the tester’s time for the more interesting and holistic tests, whether manual or automated.
Automated tester tests need to sit close to the code too
As discussed above, agile software is now deployed piecemeal-wise, and not in one big deployment bang. This means that the software testers tests need to test not just the whole big product, but also parts of it.
This means that if the microservice your team is writing is a web service that communicates via REST API, then your tests should check that REST API in isolation from the rest of the product. And if your team is building UI components that the other parts of the company uses, those components should be tested in isolation from the rest of the product.
In other words, software testers should not only do end to end tests, but should also check parts of the software, and not the whole thing. Which parts? A good rule of thumb is that if your team is building components and microservices that are part of a bigger whole, you should test those components in isolation and not only as parts of the bigger whole.
Automated tester tests need to be part of the CI process
Software developers that operate as part of a true agile process frequently use CI tools to help them cope with the fantastic pace of software development. For software developers, the CI tool isthe gatekeeper for their software change.
What is a CI tool? CI means Continuous Integration, and a CI Tool is a tool that listens to changes in the source code, builds it, runs the tests, and if the tests pass, mark the changes “green”. If they fail, they usually email the person that submitted the software changes. The CI tool helps the software developer ensure that all the tests they write are executed for those specific changes. Examples of CI Tools are Jenkins, TeamCity, Travis, Bamboo.
These are fantastic tool, and are definitely part of the shift-left movement, as the tests runs for every code change. A software developer should ensure that not only do the software developer tests run, but also their own tests. A code change needs to be green only if the developer tests pass and the tester tests pass.
The three rules above are, for me, the essence of the shift-left movement. To be agile, automated tests should not be a separate stage of the development, but should be part and parcel of it. And to implement this should be the responsibility of both the tester and the developer.
And this responsibility brings us to what is for me, the holy grail of the shift-left movement:
Automated tester tests need to be part of the source code
There should be no difference between where the developer tests reside and where the tester tests reside. They should be part of the same source code, and as we said above, run as part of the same CI process. I will dedicate a blog post in the near future to this important idea (“Speaking the language of frontend developers”).
Remember, in an agile process, the tests become as important and integral as the development itself. Embrace agility.
Shifting Right
But if the shift-left movement means that a tester does not check the whole product holistically, then does that mean that it does not need to be checked as a whole?
Of course it has to be checked as a whole. The big question is when? If there is no release, then when do we test the whole product? The answer may surprise you and is part of the shift-right movement.
And the answer is — whenever you want to. Once there is no release, and software changes keep pouring out of the pipe, and they are well-tested due to the shift-left movement, then you can do E2E testing whenever you want to.
Run your end to end tests continuously
If you can run your end to end tests whenever you want, then just run them continuously. Yes, source code keeps pouring in from the left, and deployments keep pouring out from the right, but that doesn’t mean that we should give up and not test. It actually means that we should run them all the time.
No, we will never be testing what is exactly going to be in production in an hour. We will always be testing an approximation. But if this run of e2e tests doesn’t uncover the bug, the next one will. And, yes, it will be too late, because that bug is already in production, but that’s OK — because we’re agile, fixing the bug also takes a very short amount of time.
In an agile world, it’s OK for bugs to sometimes reach production. This is what is meant by Facebook when they say “Move fast and break things”. While an extreme sentence, it does hold the important truth that trying to stop all bugs from reaching production doesn’t help development, it harms it.
Run your end to end tests in production
Some companies take this to the extreme, and have software testers test the full product not in a testing or staging environment, but actually in production, after it’s released! (this is why it’s called “shift right”). And incredibly enough, this is not as dumb as it sounds. Testing in production enables us to uncover bugs that are due to:
- bad data that can only be found in production
- the scale and amount of people using the application in production
- the deployment system used for production
- and bugs that are in the infrastructure (servers, networks, database, etc) used in production
Embracing Agility
So yes, embrace the agile manifesto. Agility changes and amplifies the role of the software tester. The shift left movement encourages the software developer to take on the role of the tester, and encourages the software tester to participate in the software development process, while the shift right movement encourages the software tester to take on some of the responsibility of monitoring the application.
And this is a good thing — the tester should not be a gatekeeper, a judge that is aloof and separate from the software development and deployment process. The tester should not be the bad guy saying, “thou shall not pass” and delaying the release of the process.
What the tester should be is an integral part of the software development process. And embracing agility allows them to do that.
Yes, But How?
Embrace agility. Shift left:
- Software developers need to test too: shifting tests to the left means that software developers should test their code too. Educate your consumers. Work with them to make this happen. Fortunately, ongoing education is moving software developers to this understanding.
- Automated tester tests need to sit close to the code too: stop thinking as gatekeepers and start thinking as developers. Test parts of the code and not just the final product.
- Automated tester tests need to be part of the CI process: the CI tool is the new gatekeeper, so your tests should be part of the CI.
- Automated tester tests need to be part of the source code: if you can, have your tests be part of the source code, and not a separate piece.
Embrace agility. Shift right:
- Run your end to end tests continuously: there is no release. Continuously test the “current” product, no matter whether it is not yet in production or already there.
- Run your end to end tests in production: there is no release. The best place to run your tests is in production, and you will get a ton of added benefits from it.
What’s in it for Applitools?
In the end, I am a developer advocate for Applitools, a company that builds a Visual UI Testing platform for testers and developers. So, what’s in this movement for Applitools?
Plenty. We believe visual UI testing tools are crucial for shifting left and shifting right. But that’s for another future blog post.
To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.
