Best Practices for Managing Software Bugs
No matter what kind of business technology solutions you use, you are going to run into issues managing software bugs.
Bugs are shortcomings in your system and can include errors in customization, configuration, integration, and documentation. In some cases, reported bugs can also turn out to be user errors or even gaps in requirements.
Finding bugs can be frustrating, possibly even overwhelming but understanding and having a consistent plan on how to report your bugs can help alleviate the stress and frustration once they’re found. Bugs typically will start popping up once configuration or development has been completed (after JPDs or around BST/CRP timeframe). Having an understanding and a consistent plan for the bug-reporting process can go a long way toward tracking these issues quickly and efficiently.
Before getting into how you can prepare yourself and your team to address these issues, understanding what bugs are and what kind of bugs you may experience, will guide you in efficiently and appropriately dealing with them.
Know Your Bug and How to Squash It
When you encounter a bug, you will log it into your bug management system. For most of our clients here at Stoneridge Software, we will log our bugs in DevOps (ADO) as a "Bug work item type. Creating a bug work item lets the appropriate team of people know that an issue has been found and that we need a resolution.
Bugs can typically be broken down into the following categories:
- Data issue - Relates to the master data. For example customers, vendors, and items.
- Configuration issue - Originates from system parameters or setup screens.
- Process issue - If the data and configurations are both correct, there could be incomplete information or missing steps in the process.
- System bug - A technical error or functionality not working as it is designed to.
- User error - The user did not execute the test correctly, did not understand the test case steps, or accidentally opened the bug.
- Design error - The error is due to an issue originating from the design. Examples include ambiguity, missing a specification, best practice, or bad design which did not work once implemented.
- Data Migration issue - The migrated data may be incorrect or missing something making it unusable.
- Security Issue - Securities have not been properly set up, so the users are unable to properly perform their duties.
How to Report Bugs
There are several key things you want to include in your report when you are bringing it to the attention of someone who can fix it. These include:
- The user ID # reporting bug
- Store #
- Register/Device name
- Environment
- Add any specific data – Item #, Vendor, Sales Order #, Customer #, etc.
- The scenario that’s being tested.
- Test case steps for how to repro the bug.
- Issue: What the issue is, with any need-to-know details
- Expected result - What the user expects that the program should do.
- Attach any screenshots in repro steps and attach a video if one is available, to the files.
- Link any parent or child, test case, or related work items to the bug.
Additionally, the more detailed information you include in your report can help your team deal with the issue faster. Below are some helpful tips on what to do and what not to do when filing your report.
What TO DO When Reporting Bugs
- Keep it simple while not oversimplifying - To reproduce the bug, the Triage team & Dev need to understand the exact problem and how they can reproduce the bug.
- Consider who needs to be involved - We should assign the appropriate team member (BPO or functional consultant) to retest and triage; after this is completed and they successfully repro the bug, they should then assign it to the developer. Caveat: if a developer has already been involved, please assign directly back to that developer vs. BPO/FC.
- Think about the core information - Include the essentials “What happened, what environment was the bug found, when does it happen, who found the bug, how frequently does it occur”, Steps to reproduce the bugs, specific data, Tag client team, SSI team, developer team with any details describing the issue & any need-to-know details in discussion
- Categorize & prioritize the bugs by understanding the severity & priority for submitting a bug - Link bug to parent/child work items, test cases, and any related items in ADO, Add tags & assign to the correct iteration
- Attach Evidence - Make sure to attach any screenshots & videos reproducing the bug. The more details the easier it will be for the triage/dev team to repro & find a resolution.
- Issue & Expected result - Include what the issue is with the test case and what the user expected to happen when performing the step/test case.
What NOT TO DO When Reporting Bugs
- Lack of detail in your report - We want to make sure that we’re not giving too little or not enough details but also want to make sure we don’t overcomplicate things. Using words that are not easily understood or that not everyone will understand. Understanding the severity & priorities of the bug is also important. Submitting a bug with incorrect priorities could prevent a bug from being resolved in the proper time frame.
- Submit multiple bugs in one ADO work item - We want to make sure not to include multiple bugs in one ADO work item. If we input more than 1 bug on an ADO work item, it will be easy to confuse the triage/developer team. We want to prevent missing important details by properly reporting bugs the first time.
- Create duplicate bugs - First, we will want to make sure to verify the bug has not already been reported. We can do this by doing a quick ADO search of keywords that are relevant to the issue at hand. You will want to prevent our resources from spending time trying to resolve an issue that’s already been reported and is actively being worked on. If the same bug has been closed and is still failing, reopen the bug or open a new bug & link to the previously closed bug.
- Finding New/Missed or Gaps in Requirements - Verify that the new bugs we’re reporting are not new or missed requirements. If we find that they are new, missed, or have gaps in the requirements, they need to go through a change control process with the project team and then be entered as a new requirement in ADO. Gaps are not bugs!
- Provide a lack of evidence - Not including need-to-know details, screenshots/videos will potentially hinder our resolution time. If we have screenshots, a test step recorder, or a video of you reproducing the bug, we need to include these in our ADO. The more details we provide to the triage/developer team, the better chance we have of quickly resolving the bug.
- Not report in a timely manner - Let's say we come across a bug that doesn’t seem like it will be that impactful, so we don’t report it. Then in a later testing iteration or even once we get to prod, we realize how impactful it really is – it will be more costly as far as resources and time to fix the issues than if you had reported it earlier. So, no matter what, no matter how big or how small, we need to report these bugs as we find them so we can get eyes on them to determine the importance of a resolution.
Tracking Bugs in DevOps
Chances are when you or someone on your team is reporting bugs, it is in your DevOps environment. The users who work there are typically the best-equipped people to deal with those issues and resolve them. If you aren't set up with this, the following lists can serve as a guide on how you can structure your environment to optimize bug fixes.
Bug Statuses
- New/Proposed – Tester logs the bug and assigns it to the functional lead, BA, or BSA to perform the analysis and documentation of the issue (or stays with the tester if the tester is also the functional lead). Then assigns to a designated person to prioritize and assign to a developer.
- Active – Once the developer is actively working on the bug, they should change the State to Active
- Resolved –The bug has been resolved and is awaiting confirmation / re-testing by the user who reported it (should be tested in Stage and UAT environments).
- Closed - The bug is verified as resolved by the original user and testing in UAT is completed. Bugs could also be moved to close if they’re unable to be reproduced/user error. This is an “archive” status.
Bug Severity Indicator
- Severity 1: Critical - A critical business function is not functioning correctly or is not available. A workaround is not available. The process cannot continue and/or downstream tests are blocked.
- Severity 2: High - A high-priority business function is not functioning correctly or is severely impaired. A workaround is available but is not ideal or viable long term.
- Severity 3: Medium - A non-critical business function is not functioning correctly or is severely impaired. An acceptable workaround is available.
- Severity 4: Low - All other test failures that have minimal impact to the general user population or that affect individual users will be classified as low. The solutions to these failures will be proposed and addressed if time and schedules permit and could be logged for inclusion in future releases, to be determined.
Bug Priority
The priority a bug is given in order of fixing will coincide with how severely the person reporting the bug marks it.
- 1: High Priority: Resolution needed as soon as possible. Cannot go live without a successful resolution.
- 2: Medium Priority: This cannot go live without a successful resolution, but it does not need to be addressed immediately.
- 3: Low Priority: Implementation or fixing is optional based on resources, time, and risk. If we go live without a successful resolution, document the issue in release notes as a known issue.
- 4: Lowest Priority: Tracks an issue that basically does not affect usage. (such as a small typo).
How to Close Bugs Once You Finish Successfully Retesting Them
When a bug is moved into resolved status, it should automatically be reassigned to the original submitter of the bug report. Then, that originating team member, should then retest to make sure that it passes.
Projects may vary in how they process retesting bugs. Most commonly, once the bug is retested and passes, the bug can be closed. If the test case still fails after retesting, we need to move back to active, reassign it to Dev team and leave notes explaining how it’s failing with any evidence we can provide to the Dev team.
Bug reports should be closed when the issue is fixed & passes retesting, is not reproducible, or is determined to be a user error.
Want to Learn More?
Please reach out to us! Our various teams have many years of experience identifying, dealing with, and resolving technology bugs.
Under the terms of this license, you are authorized to share and redistribute the content across various mediums, subject to adherence to the specified conditions: you must provide proper attribution to Stoneridge as the original creator in a manner that does not imply their endorsement of your use, the material is to be utilized solely for non-commercial purposes, and alterations, modifications, or derivative works based on the original material are strictly prohibited.
Responsibility rests with the licensee to ensure that their use of the material does not violate any other rights.