CI/CD & Test Automation for Dynamics 365 in Azure DevOps/VSTS – Part 1

In this blog series, we will explore building out DevOps processes and practices for Dynamics 365 Customer Engagement (CE) by utilizing Wael Hamez MSCRM Build tools. In this first blog, we will cover the version control for Solutions.

What is DevOps?

DevOps is a new term emerging from the collision of two major related trends. The first was also called “agile infrastructure” or “agile operations”; it sprang from applying Agile and Lean approaches to operations work.  The second is a much-expanded understanding of the value of collaboration between development and operations staff throughout all stages of the development lifecycle when creating and operating a service, and how important operations has become in our increasingly service-oriented world (cf. Operations: The New Secret Sauce).

Problem Statement

I have started working on CI/CD when I was assigned as a Platform Engineer for a Dynamics 365 CE implementation project. At that time, I had a few key challenges with Dynamics 365 CE. I have listed those below:

  • Solution files were manually extracted and imported to target as a deployment process
  • No Unit testing or validation for deployed solution
  • Multiple deployment process is followed between release environments. For example, in Dev and Sit environment, the solution was migrated manually, and in UAT, Pre-Prod and Prod environment DB compare was applied to promote changes
  • Master data were mutually entered in each environment
  • Multiple developers working in the same organizations overwriting the changes.

Before we start solving the problem, let us take a moment to define some of the mandatory steps which we need to follow.

  • All the source code must be in source control. For example, Plugin Code, Custom workflows, actions, Warehouses (HTML, js, etc) Solution file, master Data, and user roles, etc.
  • Check-in regularly and every change should trigger the commit process
  • Commit process should be short and validates the committed component(Gated-Checkin)
  • Track changes and rollback as needed

Pre-Requisites

Here is a little bit of information regarding the environment.  We are using Dynamics 365 (Online) v9.0 and VSTS/Azure DevOps for version control, build and deployment.  I am using Visual Studio 2017 with the Microsoft Dynamics CRM SDK Templates extension installed.

I have also installed the Dynamics 365 Build Tools(By Wael Hamze) into our VSTS Organization so the tasks are available for all Build and Release Definitions.

Version control for solutions

Solutions in Dynamics 365 CE are in essence a package containing any customization we’ve done to our environment that we can export from one environment then import into various environments. When exported from an environment, solutions are in the form of a zip file. When that zip file is unzipped, the output directory contains folders for plugins, web resources, and any workflows we have made as well as XML files defining the schema of any customization we have done. In the zipped format, our schema definition is contained in one massive file. Consider this zip file as a binary, or in layman’s terms, a tidy package with a fancy bow, i.e. It may look nice but it’s not easy to see what’s inside and it’s a poor format for version control.

Solution packager is a tool that essentially takes our Dynamics 365 CE solution zip file and breaks it out into a logical folder structure by decomposing the contents. The resulting output shows a more granular view of our solution and is considerably more friendly for version control as you can see from the example screenshots below.

Dynamics 365 Instance Management

Below are the little regarding how we have managed the dynamics 365 instances for achieving this DevOps.

  1. Dev environment: It is the true source for all the customization and configuration changes.
  2. SIT: where you test all the functionality against different types of data by internal test users.
  3. UAT: where you test all the functionality against different types of data by business users.
  4. Production: Go Live Instance.

Dynamics CRM CICD process

The below diagram illustrates the basic flow of Dynamics 365 CE Workflow process.

In this part, we are going to see the highlighted one in detail.

The process followed by the developer is essentially these steps (pictured below), the two steps in the middle are handled by the SolutionExportDev.cmd script:

Items

It will export the solution from source instance and unpack the solutions into the folders in BuildAutomation.Solutions

  • Developers benefit using this process by making the solution deployment repeatable and predictable.
  • Developers can associate their work items with their check-ins
  • Very useful for tracking changes (traceability) and comparing solution files
  • SolutionExportDev.cmd – This PowerShell script connects to Dynamics where the developers are customizing their solution, downloads the zip file to the developer’s workspace and extracts the Unmanaged solution into the BuildAutomation.Solutions /BuildAutomation folder.  The Core Tools folder contains the Solution Packager tool from the Dynamics Core Tools NuGet package (pictured below)

Our solution consists of a CRM Deployment Packager project(BuildAutomation.SolutionPackager) and Solutions(BuildAutomation.Solutions) which is used as a container for the extracted files and folders of the solution. 

We added a Scripts(SolutionExportDev.cmd) folder to the project (BuildAutomation.SolutionPackager) which contains some below PowerShell scripts.

PowerShell Script:
set crmuserpwd=%3
set crmuser=%2
set crmserver=%4
set orgname=%1 
set crmuserdomain=%8%
set solution=%5
set folder=%6
set auth=%7
if "%solution%"=="" set solution=BuildAutomation
if "%folder%"=="" set folder=%solution%
if "%auth%"=="" set auth=ifd
set managed=false
set packagetype=unmanaged
set projectdir=BuildAutomation.Solutions
bin\Debug\BuildAutomation.SolutionPackager export managed=%managed% allowdelete=true crmuser=%crmuser% auth=%auth% crmuserpwd=%crmuserpwd% crmserver=%crmserver% orgname=%orgname% solution=%solution% solutionfile=..\%projectdir%\%folder%\%solution%.zip 
echo Exit Code is %errorlevel%
IF %ERRORLEVEL% EQU 0 Echo Solution export was SUCCESSFUL
IF %ERRORLEVEL% EQU 1 (
   echo Aborting the process, as solution export has FAILED with ErrorCode %errorlevel%
   exit /b %errorlevel%
)
tools\solutionpackager.exe /action:Extract /packagetype:%packagetype% /zipfile:..\%projectdir%\%folder%\%solution%.zip /folder:..\%projectdir%\%folder%\ /allowDelete:No
exit /b
:EXIT
@pause

Developers need to do below the configuration in visual studio(Run the command prompt from inside Visual Studio) to execute the SolutionExportDev.cmd

To make the tool available, add it to the external tools list. Here are the steps:

  1. Open Visual Studio.
  2. Select the Tools menu, and then choose External Tools.
  • On the External Tools dialog box, choose the Add button. A new entry appears.
  • Enter a Title for your new menu item such as Command Prompt.
  • In the Command field, specify the file you want to launch, such as SolutionExportDev.cmd
  • In the Arguments field, specify the organization name, CRM username, password, server name, and domain name
  • Choose a value for the Initial directory field, such as Project Directory.

Example:

Title – Name of the external tools to run

Command – Provide the physical path to the command file

Arguments – SolutionExportDev.cmd accepts 4 arguments

  • Organization Name
    1. Username
    2. Password
    3. CRM Server name
    4. Domain

Initial directory – Provide the directory path of the SolutionExportDev.cmd file.

Use Output Window – Enable

Prompt for Arguments ­– Disable

Close on exit–Enable

  • Choose the OK button.

The new menu item is added, and you can access the command prompt from the Tools menu as follows:

To run this, please click on ExportCrmSolution. It will export the solution from source instance and unpack the solutions into the folders in BuildAutomation. Solutions.

Once the above step is done, the developer now has some manual steps to perform depending on the files/changes that were extracted from the solution:

  1. Include any new file under the extracted folder Solutions\BuildAutomation (extracted) into the project- BuildAutomation. Solutions (so that they can be checked-in).
    NOTE: When doing the check-in, be sure to check the ‘Excluded Changes’ if additional files were detected and promote them to the ‘Included Changes’ as needed.
  2. For any .xaml file (under Solutions\BuildAutomation\Workflows or Solutions\BuildAutomation \Entities\xxxx\Formulas), make sure that Build Action is ‘None’ (not ‘Page’ or ‘XamlAppDef’). If this step is skipped, you may get one of this compilation error (s):
    – Project file must include the .NET Framework assembly ‘WindowsBase, PresentationCore, PresentationFramework’ in the reference list.
    – Assembly “System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a’ cannot be resolved”.
  • Ensure that DLL files under Solutions\..\PluginAssemblies are added to source control.
  • Undo all pending changes that do not specifically apply to your changes.
  • Associate the check-in with a work item(s), write a comment and check-in all required changes.

Note: Check-in to TFS will auto trigger the deployment process.

In my next blog, we will see how to pack the solution and deploy it in the Target instance using VSTS Build and Release definition.

If you are interested in this topic and would like to do some further self-study I encourage you to check out my blog on this.

One thought on “CI/CD & Test Automation for Dynamics 365 in Azure DevOps/VSTS – Part 1

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s