So, you may be asking yourself, "FOIL, isn't that a math term?" Yes it is; however, you can use a similar FOIL method for requirements testing too.
F – FUNDAMENTALS
"Do the Fundamentals work as written or intended by design?"
F = Fundamentals: The very first thing you must do with a requirement is verify that it is working as expected within a logical confine of itself. It's a fundamental need to prove this works as intended for the basis and the foundation of the application under test as a whole. Without doing this first, you can't proceed to finding out how it doesn't work; which brings us to the next point….
O – OUTSIDE
"How gracefully do the requirements handle events when going outside the logical confines of the requirement?"
O=Outer/Outside of the requirement: This is when we will most likely encounter a bug because this is usually where no one expects you to look, especially your programmers. For example:
- Imagine you are testing an application that allows you to enter any whole numbers 1-9 in a textbox that only supports integers.
- You verify this works correctly from the "Fundamental" level by entering any random number from 1-9 to ensure at least some integer works
- Once you've verified the requirement "works", now it's time to test the outside.
- In this example, the outside consists of "0, 10, -1, 0.1, 1.2". This will prove, if there are no issues here, that there is most likely nothing that will work outside of integers 1-9.
- There are more ways to test the outside, however, there is a limit to how much "outside" should be done. You need to limit this type of testing scrutiny or attack to a science. Attack the outside in places you feel would most likely produce an issue; or, in better ways, how an end-user could accidentally incorrectly enter data (like an accidental space in front of a valid number like "2").
I – INTEGRATION
"Does the requirement or feature work consistently with the rest of the application?"
I =Integration: After looking into the fundamentals and outside, you need to observe how well this function or feature works with the rest of the application. Integration is the act of making sure that new functionality, features, requirements, etc. don't impede the previously working functionality. This also means that there is collaborative functionality between the old and new functionality.
This is also when you check to make sure this new function or feature is consistent with the rest of the application. For example:
Imagine you're working in sprint 3 on an agile project, or you're somewhere near the middle of your second milestone in a waterfall project. You begin testing for the requirement and you notice the color of the button is different and the font doesn't look right in comparison to what all other buttons look like, or if what you're looking at isn't a button but an entire page. You notice the background is a different color than the rest of the screens or that the orientation of the text within the screen isn't the same as all other pages. There are limitless inconsistencies that can occur through the lifecycle of a project.
You will most likely find a bug with consistency sometime throughout the lifecycle of any project. This is especially common if you have a team that changes members over time and/or a project that takes months to finish. Someone new is likely to not realize 100% how it was done before or exactly what was used for an asset/reference/class/etc. The only thing "consistent" about inconsistencies is that inconsistencies are consistent.
L – LIST
L= List: The last thing you do is list (log bugs) for any and all issues you found while doing the first 3 steps. While writing the bugs, you need to be 100% sure you know how to reproduce what you found. It's best to always reproduce the bugs you found while running through the first 3 steps before listing your bugs.
While you report your issue, you need:
- A specific title to give a general idea of the bug like:
- "There are no validation checks for numbers beyond 9"
- Describe your title a little more (where clarity may be needed) :
- "While making sure that no numbers beyond 9 were considered valid by the application, I entered the number "10" and found out the number was accepted by the application. I did notice, however, when I entered "0", the application returned the expected validation"
- For issues that are more complicated, attaching pictures and/or videos to the bug will be necessary to inform the team of what is going on with the issue.
- Now, it's time to describe the steps you took to reproduce the issue in the best detail you can:
- Launch the Numbers application – Version "whatever"
- Enter "10" as the number you're trying to enter into the field
- Click "go"/ "enter" (whatever the method of proceeding is)
- Observe there is no validation check on the application for "10"
- After entering the steps to reproduce the issue, it's time to enter the expected result:
- "Entering any number beyond 9 gives a validation call back to the end-user saying 'This is an invalid number. The number must be between 1 to 9. The number you entered is too high. Try again.'"
- Lastly, and where applicable, make sure the bug is assigned to or associated with something in your team environment (like Team Foundation Server), so someone will be able to see the bug to work on it.
Rinse and repeat as needed.
In my experience, this is probably one of the more effective processes one can do. This is simple enough to apply to almost anything and it's how I normally approach requirements. What you test and how you test it all depends on your time and the risks involved.
If you'd like to hear more about how Sparkhound's certified Quality Assurance Analysts can help you with your software project; please reach out to us. We'd love to help!