Successful Testing Plans for Dynamics 365 Finance and Supply Chain Management Updates
Dynamics 365 Finance and Supply Chain Management typically offers six to eight updates per year that include new features, functionality, bug fixes, and other enhancements for the system. Microsoft allows you to skip up to three updates, which translates to roughly two updates per year minimum. This allows you more flexibility to plan updates around team availability. Testing every release prior to applying it to your environment is important in ensuring critical business processes run smoothly after updating, as well as reducing the risk of broken functionality from feature code conflicts. Below I've outline the important things you must consider for testing each release for Dynamics 365 Finance and Supply Chain.
Identifying In-Scope Processes
Regression testing should focus on important areas impacted by the update, especially if a failure in that area would halt operations. To determine importance, consider transaction volume, the sensitivity of the information, difficulty detecting or correcting errors, and possible fines if that area is compromised. Once important areas are identified, focus your testing on these areas and determine the size of your testing scope. Depending on the criticality of the process you may need to test the upstream and downstream areas from that process as well.
Plan Testing
You will be informed of a new update about two weeks before the GA, or general availability, date. This leaves you with only around two to four weeks to test before it is applied to production (depending on your preferences set up in LCS). Determine the team lead, team members, and time frame needed to execute tests and ensure everyone is prepared for testing when the update is applied. Next, select and implement a test management tool such as DevOps to manage your list of tests, test assignments, and pass / fail results. Finally, develop a risk mitigation strategy for areas that cannot be tested, such as certain ISVs.
Considerations when planning testing activities:
- Determine who will update the test system’s data and code.
- Make sure all (non-preview) features are enabled when you get an update.
- Update any configurations as needed, e.g. copied from production but not valid for test.
- Execute testing and record results.
- Report results to stakeholders.
- If successful update production, or if errors were found request a postponement from Microsoft.
Microsoft publishes the dates for the next few updates here, you can use this to plan your next update time frame.
Create the Tests
Prioritize tests in order of importance and execute the most important tests first. If possible, group tests together strategically to increase efficiency. Next, scripts should be written to ensure your input is the same every time you test. This is important for continuity and consistency purposes, especially if doing manual testing as different team members may be performing different rounds of testing. Finally, determine which tests are eligible for automation and automate as much as possible. Automated tests are more efficient, can be done outside of business hours, have more rapid feedback, are executed consistently, and free up business resources. After creating tests, it is time to execute them.
Execute Testing
Since there are typically around 6 months between regression tests (if taking updates twice yearly), hold a “Kickoff” session to remind the team of the testing processes, procedures, and how to use the test management system. Kickoff should be spent reviewing the cumulative release notes with the testing team, recapping the timeline and expectations, explaining where to find the list of tests, and explaining how and where to report results.
Dynamics users have options to get a head start on testing. The PEAP (Preview Early Access Program) has a low barrier of entry and releases updates about six weeks before the GA date. PEAP releases are not final and features can be added or removed. The release validation program allows you to share databases, tests, and code with Microsoft. Microsoft then runs automated tests and investigates results. Finally, the first release program makes you among the first customers to receive updates to your production environment along with dedicated support. More information on how to join these programs can be found here.
Automated Testing Tools
RSAT (Regression Suite Automation Tool) is free with Dynamics 365 Finance and Supply Chain Management. RSAT offers basic automated testing via integration with DevOps and D365.
If you are looking for a more robust automated testing tool, Executive Automats is for you. Executive Automats is a best-of-breed automation tool design specifically for the Finance and Supply Chain Management space. It offers a variety of functions over RSAT, including performance testing, looping, if-then condition testing, and RPA (robotic process automation).
Automation Best Practices
- If possible, break tests into small chunks for easier maintenance. This simplifies recording and updating the test for process changes.
- If using RSAT, leverage Excel formulas to randomize data.
- Control all possible variables by setting up master data, resetting usage data, and resetting personalizations before recording and playback. This lowers the probability of an artificial failure when the test is played back.
Updating your system gives you access to the latest D365 Finance and Supply Chain Management features.
Under the terms of this license, you are authorized to share and redistribute the content across various mediums, subject to adherence to the specified conditions: you must provide proper attribution to Stoneridge as the original creator in a manner that does not imply their endorsement of your use, the material is to be utilized solely for non-commercial purposes, and alterations, modifications, or derivative works based on the original material are strictly prohibited.
Responsibility rests with the licensee to ensure that their use of the material does not violate any other rights.