To provide services at the highest level, we use cookies as described in our cookie policy. By using our website, cookies will be placed on your device. You can change your browser settings at any time. Check our cookie policy.
Authors
- Krzysztof Grabowski
Authors
- Krzysztof Grabowski
Content
Categories
Introduction
Dear Reader,
Welcome to a new series of articles dedicated to the world of testing.
In this series, we will delve into the intricacies of the work of testers/QA professionals and break down what happens during the testing process, the main principles guiding testers, their approaches to various situations, and the primary goal of their work.
Why is it essential to understand more about the testing process? If you work in the IT industry but aren't necessarily involved in technical aspects, you might have had little contact with testing. It might seem straightforward. However, like anything else, it can be more complex upon closer inspection. It's worth learning about specific issues and principles to understand them better.
This matter is more straightforward for developers, as they constantly test their code. However, for those working more on the business or project management side, this series can be instrumental in understanding how testing impacts the entire project lifecycle.
Firstly, we'll discuss the seven main principles of testing. This is the tester's decalogue, yet project managers often need to pay more attention to these fundamental concepts, leading to erroneous assumptions.
Welcome to a new series of articles dedicated to the world of testing.
In this series, we will delve into the intricacies of the work of testers/QA professionals and break down what happens during the testing process, the main principles guiding testers, their approaches to various situations, and the primary goal of their work.
Why is it essential to understand more about the testing process? If you work in the IT industry but aren't necessarily involved in technical aspects, you might have had little contact with testing. It might seem straightforward. However, like anything else, it can be more complex upon closer inspection. It's worth learning about specific issues and principles to understand them better.
This matter is more straightforward for developers, as they constantly test their code. However, for those working more on the business or project management side, this series can be instrumental in understanding how testing impacts the entire project lifecycle.
Firstly, we'll discuss the seven main principles of testing. This is the tester's decalogue, yet project managers often need to pay more attention to these fundamental concepts, leading to erroneous assumptions.
1. Testing Reveals Defects but Cannot Prove Their Absence
That's right. It sounds like a good excuse. We start testing but immediately note that it's not our fault if additional bugs are found. What's wrong with you people?
When something doesn't work, the question often arises, "Who tested this?" or "Was it tested at all?" It always seems obvious that if something has passed testing, it must be error-free. However, it's a bit more complex and worth discussing openly.
Yes, testing reveals defects, but one can never be sure that all defects have been found. The longer a product is tested, the less likely defects still exist. At some point, one might start believing that everything is fine. But can we be sure we've missed nothing? There's always a chance of hidden bugs that might not appear in a given software version at a given time, potentially surfacing later, like the "Y2K bug," during daylight saving time changes, hardware changes, etc.
When we start testing a product, the frequency of finding defects is much higher at the beginning than at the end.
Eventually, a decision has to be made - "It's OK." Depending on the complexity of the tests and the amount of material to be tested, this could be a matter of minutes or hours of work. Given a limited budget, a reasonable approach is necessary. Therefore, we follow a simple rule: we test as long as we find new errors. Once all test scenarios have been covered and no new issues arise, it's time to stop. This is akin to risk management. There might still be something to find, or there might not. Is it worth the client's money to pay for another hour of testing when everything looks good? A good tester can often sense these things intuitively, and it's worth trusting their intuition.
When something doesn't work, the question often arises, "Who tested this?" or "Was it tested at all?" It always seems obvious that if something has passed testing, it must be error-free. However, it's a bit more complex and worth discussing openly.
Yes, testing reveals defects, but one can never be sure that all defects have been found. The longer a product is tested, the less likely defects still exist. At some point, one might start believing that everything is fine. But can we be sure we've missed nothing? There's always a chance of hidden bugs that might not appear in a given software version at a given time, potentially surfacing later, like the "Y2K bug," during daylight saving time changes, hardware changes, etc.
When we start testing a product, the frequency of finding defects is much higher at the beginning than at the end.
Eventually, a decision has to be made - "It's OK." Depending on the complexity of the tests and the amount of material to be tested, this could be a matter of minutes or hours of work. Given a limited budget, a reasonable approach is necessary. Therefore, we follow a simple rule: we test as long as we find new errors. Once all test scenarios have been covered and no new issues arise, it's time to stop. This is akin to risk management. There might still be something to find, or there might not. Is it worth the client's money to pay for another hour of testing when everything looks good? A good tester can often sense these things intuitively, and it's worth trusting their intuition.
2. Exhaustive Testing is Impossible
It's impossible to test everything—every combination, all data, and every possible outcome. In practice, we verify as many scenarios as possible using boundary values of conditions that can be triggered in the application. We check the correctness of the application's basic positive and negative behaviors to ensure that these key values trigger appropriate actions. Testing everything comprehensively would be an inefficient use of the budget, so to protect the client's finances, one must do the testing sparingly.
To ensure the client gets a well-functioning product, it's essential to test, question the system's correct operation, and prove to oneself that it works correctly. However, like everything else, it requires a balanced approach. If 10,000 different numerical combinations are possible, will we conduct 10,000 separate test cases? Of course not, as it's physically unfeasible, and we must be fully aware of that.
To ensure the client gets a well-functioning product, it's essential to test, question the system's correct operation, and prove to oneself that it works correctly. However, like everything else, it requires a balanced approach. If 10,000 different numerical combinations are possible, will we conduct 10,000 separate test cases? Of course not, as it's physically unfeasible, and we must be fully aware of that.
3. Early Testing Saves Time and Money
One could write a separate article or even a book on this topic. Testing can begin from the very start of product development. One can test the concept itself for flaws. One can test documentation for inconsistencies. How much cheaper is it to correct a few sentences in a text file than to have a team of programmers rebuild the code?!
A tester involved from the beginning can catch specific issues early, which are much cheaper to fix at that stage. By validating early mockups and transition diagrams and assessing usability, we can eliminate problems before they arise. Unfortunately, the approach of testing only a functioning product still prevails. However, early investment in testing can bring significant savings later in the project. The more complex the application code becomes, the more costly and complicated its modification.
Testers are usually engaged at the final production stage for the sake of the project's budget. Still, this approach comes from waterfall methodologies. Mature organizations using modern methodologies know that early testing pays off, and studies show a reduced time spent fixing errors by up to 70%.
A tester involved from the beginning can catch specific issues early, which are much cheaper to fix at that stage. By validating early mockups and transition diagrams and assessing usability, we can eliminate problems before they arise. Unfortunately, the approach of testing only a functioning product still prevails. However, early investment in testing can bring significant savings later in the project. The more complex the application code becomes, the more costly and complicated its modification.
Testers are usually engaged at the final production stage for the sake of the project's budget. Still, this approach comes from waterfall methodologies. Mature organizations using modern methodologies know that early testing pays off, and studies show a reduced time spent fixing errors by up to 70%.
4. Defects Tend to Cluster Together
This simple yet true principle is repeatedly confirmed in practice: some components or software modules may contain more defects or be responsible for more failures than others. Defects tend to cluster and escalate problems in the rest of the application. This occurs due to various factors, such as different parts of the application being developed by others, varying complexity levels, etc. As a result, defects are not evenly distributed across the system but occur in clusters. If a tester starts finding defects in one area, they will focus more on that area.
The more defects they find, the easier it is for developers to locate the trustworthy source of the failures. This might be more complex than just fixing one defect that initially doesn't seem like part of a bigger problem.
Applications should be tested sparingly but focused on the most problematic modules.
The more defects they find, the easier it is for developers to locate the trustworthy source of the failures. This might be more complex than just fixing one defect that initially doesn't seem like part of a bigger problem.
Applications should be tested sparingly but focused on the most problematic modules.
5. The Pesticide Paradox
This principle states that if we test an application uniformly, over time, defects that are not detectable by basic test scenarios might appear, slipping through tests and revealing themselves during more complex operations. Automated testing suffers in this context. Tests might pass "green," but a defect causing a failure exists in the system and remains undetected. Thus, manual testing is more accurate in the long run despite being less efficient and more time-consuming. A human-approaching test should have a fresh perspective each time and often detect defects in subsequent encounters with the application, not the first visit - although we all wish it were so. Each time, we approach tests differently and invent new data.
This principle is multifaceted. It suggests not testing an application uniformly, but also highlights the importance of diverse approaches to testing and how detrimental strict adherence to test scenarios can be.
The human approach is crucial. The routine can creep in during repeated testing of the same thing, making it easy to lose vigilance. Hence, it's vital to maintain and remember the childlike curiosity within us, always trying something new and playing detective. As we enter the AI era, we must be fully aware of our distinguishing human qualities - creativity and ingenuity, non-schematic actions, and being a continuous source of ideas. In testing, this is very significant.
This principle is multifaceted. It suggests not testing an application uniformly, but also highlights the importance of diverse approaches to testing and how detrimental strict adherence to test scenarios can be.
The human approach is crucial. The routine can creep in during repeated testing of the same thing, making it easy to lose vigilance. Hence, it's vital to maintain and remember the childlike curiosity within us, always trying something new and playing detective. As we enter the AI era, we must be fully aware of our distinguishing human qualities - creativity and ingenuity, non-schematic actions, and being a continuous source of ideas. In testing, this is very significant.
6. Testing Depends on Context
The methods and types of tests used can vary depending on the type and context of the application or system. Only some things are tested the same way because not everything being tested is built the same way. Initially, the principle seems simple - test what has been built to confirm its complete and correct operation - but also test with the user in mind and the system's significance - what is most important in the given application.
We will focus very little on visual aspects when dealing with an admin panel, but the end-user part must be perfect. We will test a web application designed for people with disabilities that differs from an art gallery application. Different things matter in an application handling financial transactions than in a blog where users are just recipients.
Testing depends on the context, and everything is tested differently. There are no identical applications, and there are no identical ways to test everything.
We will focus very little on visual aspects when dealing with an admin panel, but the end-user part must be perfect. We will test a web application designed for people with disabilities that differs from an art gallery application. Different things matter in an application handling financial transactions than in a blog where users are just recipients.
Testing depends on the context, and everything is tested differently. There are no identical applications, and there are no identical ways to test everything.
7. Absence-of-Errors Fallacy
Another multifaceted principle. It partly refers to the first principle - that we can never be sure there are no errors. Even if a product works flawlessly and has passed tests that found nothing, it is still just a state for that moment, in a specific version of the application, on a particular day, on a given device. It's constantly being determined how the application will behave the next day when the date changes, the operating system version changes, the browser version is updated, etc.
Another interesting interpretation of this principle is that technical correctness alone may not be sufficient to determine the product's success.Testing does not focus solely on the technical side but can extend much wider. For example, we can test the application's usability, where different types of “errors” might still be present, the flow isn't smooth enough, or the interface isn't clear enough for the user.
An application that is technically sound isn't necessarily one that functions well.
Another interesting interpretation of this principle is that technical correctness alone may not be sufficient to determine the product's success.Testing does not focus solely on the technical side but can extend much wider. For example, we can test the application's usability, where different types of “errors” might still be present, the flow isn't smooth enough, or the interface isn't clear enough for the user.
An application that is technically sound isn't necessarily one that functions well.
Conclusion
Testers are familiar with these seven principles and constantly remember them. These principles guide them in their daily activities. Project managers must also be aware of them to understand the testing process and product quality better.
These principles also show that tests can't guarantee anything 100%. It's worth testing and eliminating issues that can and should be fixed, but one can never be fully certain of anything or what the future holds.
Therefore, it's crucial to maintain and support the product continuously. You never know when new software issues might arise, and having a trusted partner who can react immediately and start repairs is extremely important. Maintaining your development team might be a good idea. Still, if we're lucky and there are few errors, it might prove unnecessary and costly - hence, having a trusted partner who can step in on demand might be the ideal much cheaper solution.
These principles also show that tests can't guarantee anything 100%. It's worth testing and eliminating issues that can and should be fixed, but one can never be fully certain of anything or what the future holds.
Therefore, it's crucial to maintain and support the product continuously. You never know when new software issues might arise, and having a trusted partner who can react immediately and start repairs is extremely important. Maintaining your development team might be a good idea. Still, if we're lucky and there are few errors, it might prove unnecessary and costly - hence, having a trusted partner who can step in on demand might be the ideal much cheaper solution.