CI/CD & Test Automation for Dynamics 365 in Azure DevOps/VSTS – Part 1

In this blog series, we will explore building out DevOps processes and practices for Dynamics 365 Customer Engagement (CE) by utilizing Wael Hamez MSCRM Build tools. In this first blog, we will cover the version control for Solutions.

What is DevOps?

DevOps is a new term emerging from the collision of two major related trends. The first was also called “agile infrastructure” or “agile operations”; it sprang from applying Agile and Lean approaches to operations work.  The second is a much-expanded understanding of the value of collaboration between development and operations staff throughout all stages of the development lifecycle when creating and operating a service, and how important operations has become in our increasingly service-oriented world (cf. Operations: The New Secret Sauce).

Problem Statement

I have started working on CI/CD when I was assigned as a Platform Engineer for a Dynamics 365 CE implementation project. At that time, I had a few key challenges with Dynamics 365 CE. I have listed those below:

  • Solution files were manually extracted and imported to target as a deployment process
  • No Unit testing or validation for deployed solution
  • Multiple deployment process is followed between release environments. For example, in Dev and Sit environment, the solution was migrated manually, and in UAT, Pre-Prod and Prod environment DB compare was applied to promote changes
  • Master data were mutually entered in each environment
  • Multiple developers working in the same organizations overwriting the changes.

Before we start solving the problem, let us take a moment to define some of the mandatory steps which we need to follow.

  • All the source code must be in source control. For example, Plugin Code, Custom workflows, actions, Warehouses (HTML, js, etc) Solution file, master Data, and user roles, etc.
  • Check-in regularly and every change should trigger the commit process
  • Commit process should be short and validates the committed component(Gated-Checkin)
  • Track changes and rollback as needed

Pre-Requisites

Here is a little bit of information regarding the environment.  We are using Dynamics 365 (Online) v9.0 and VSTS/Azure DevOps for version control, build and deployment.  I am using Visual Studio 2017 with the Microsoft Dynamics CRM SDK Templates extension installed.

I have also installed the Dynamics 365 Build Tools(By Wael Hamze) into our VSTS Organization so the tasks are available for all Build and Release Definitions.

Version control for solutions

Solutions in Dynamics 365 CE are in essence a package containing any customization we’ve done to our environment that we can export from one environment then import into various environments. When exported from an environment, solutions are in the form of a zip file. When that zip file is unzipped, the output directory contains folders for plugins, web resources, and any workflows we have made as well as XML files defining the schema of any customization we have done. In the zipped format, our schema definition is contained in one massive file. Consider this zip file as a binary, or in layman’s terms, a tidy package with a fancy bow, i.e. It may look nice but it’s not easy to see what’s inside and it’s a poor format for version control.

Solution packager is a tool that essentially takes our Dynamics 365 CE solution zip file and breaks it out into a logical folder structure by decomposing the contents. The resulting output shows a more granular view of our solution and is considerably more friendly for version control as you can see from the example screenshots below.

Dynamics 365 Instance Management

Below are the little regarding how we have managed the dynamics 365 instances for achieving this DevOps.

  1. Dev environment: It is the true source for all the customization and configuration changes.
  2. SIT: where you test all the functionality against different types of data by internal test users.
  3. UAT: where you test all the functionality against different types of data by business users.
  4. Production: Go Live Instance.

Dynamics CRM CICD process

The below diagram illustrates the basic flow of Dynamics 365 CE Workflow process.

In this part, we are going to see the highlighted one in detail.

The process followed by the developer is essentially these steps (pictured below), the two steps in the middle are handled by the SolutionExportDev.cmd script:

Items

It will export the solution from source instance and unpack the solutions into the folders in BuildAutomation.Solutions

  • Developers benefit using this process by making the solution deployment repeatable and predictable.
  • Developers can associate their work items with their check-ins
  • Very useful for tracking changes (traceability) and comparing solution files
  • SolutionExportDev.cmd – This PowerShell script connects to Dynamics where the developers are customizing their solution, downloads the zip file to the developer’s workspace and extracts the Unmanaged solution into the BuildAutomation.Solutions /BuildAutomation folder.  The Core Tools folder contains the Solution Packager tool from the Dynamics Core Tools NuGet package (pictured below)

Our solution consists of a CRM Deployment Packager project(BuildAutomation.SolutionPackager) and Solutions(BuildAutomation.Solutions) which is used as a container for the extracted files and folders of the solution. 

We added a Scripts(SolutionExportDev.cmd) folder to the project (BuildAutomation.SolutionPackager) which contains some below PowerShell scripts.

PowerShell Script:
set crmuserpwd=%3
set crmuser=%2
set crmserver=%4
set orgname=%1 
set crmuserdomain=%8%
set solution=%5
set folder=%6
set auth=%7
if "%solution%"=="" set solution=BuildAutomation
if "%folder%"=="" set folder=%solution%
if "%auth%"=="" set auth=ifd
set managed=false
set packagetype=unmanaged
set projectdir=BuildAutomation.Solutions
bin\Debug\BuildAutomation.SolutionPackager export managed=%managed% allowdelete=true crmuser=%crmuser% auth=%auth% crmuserpwd=%crmuserpwd% crmserver=%crmserver% orgname=%orgname% solution=%solution% solutionfile=..\%projectdir%\%folder%\%solution%.zip 
echo Exit Code is %errorlevel%
IF %ERRORLEVEL% EQU 0 Echo Solution export was SUCCESSFUL
IF %ERRORLEVEL% EQU 1 (
   echo Aborting the process, as solution export has FAILED with ErrorCode %errorlevel%
   exit /b %errorlevel%
)
tools\solutionpackager.exe /action:Extract /packagetype:%packagetype% /zipfile:..\%projectdir%\%folder%\%solution%.zip /folder:..\%projectdir%\%folder%\ /allowDelete:No
exit /b
:EXIT
@pause

And then update the Program.cs file of BuildAutomation.SolutionPackager as follows:

namespace BuildAutomation.Crm.SolutionPackager
{
    using System;
    using System.Globalization;
    using System.IO;
    using System.Linq;
    using System.Xml;
    using Microsoft.Crm.Sdk.Messages;
    using Microsoft.Xrm.Sdk;
    using Microsoft.Xrm.Sdk.Messages;
    using Microsoft.Xrm.Sdk.Metadata;
    using Microsoft.Xrm.Sdk.Query;
    using BuildAutomation.Common;

    #region Component Types
    public enum ComponentType
    {
        [System.Runtime.Serialization.EnumMemberAttribute]
        None = 0,

        [System.Runtime.Serialization.EnumMemberAttribute]
        Entity = 1,

        [System.Runtime.Serialization.EnumMemberAttribute]
        Attribute = 2,

        [System.Runtime.Serialization.EnumMemberAttribute]
        Relationship = 3,

        [System.Runtime.Serialization.EnumMemberAttribute]
        AttributePicklistValue = 4,

        [System.Runtime.Serialization.EnumMemberAttribute]
        AttributeLookupValue = 5,

        [System.Runtime.Serialization.EnumMemberAttribute]
        ViewAttribute = 6,

        [System.Runtime.Serialization.EnumMemberAttribute]
        LocalizedLabel = 7,

        [System.Runtime.Serialization.EnumMemberAttribute]
        RelationshipExtraCondition = 8,

        [System.Runtime.Serialization.EnumMemberAttribute]
        OptionSet = 9,

        [System.Runtime.Serialization.EnumMemberAttribute]
        EntityRelationship = 10,

        [System.Runtime.Serialization.EnumMemberAttribute]
        EntityRelationshipRole = 11,

        [System.Runtime.Serialization.EnumMemberAttribute]
        EntityRelationshipRelationships = 12,

        [System.Runtime.Serialization.EnumMemberAttribute]
        ManagedProperty = 13,

        [System.Runtime.Serialization.EnumMemberAttribute]
        Role = 20,

        [System.Runtime.Serialization.EnumMemberAttribute]
        RolePrivilege = 21,

        [System.Runtime.Serialization.EnumMemberAttribute]
        DisplayString = 22,

        [System.Runtime.Serialization.EnumMemberAttribute]
        DisplayStringMap = 23,

        [System.Runtime.Serialization.EnumMemberAttribute]
        Form = 24,

        [System.Runtime.Serialization.EnumMemberAttribute]
        Organization = 25,

        [System.Runtime.Serialization.EnumMemberAttribute]
        SavedQuery = 26,

        [System.Runtime.Serialization.EnumMemberAttribute]
        Workflow = 29,

        [System.Runtime.Serialization.EnumMemberAttribute]
        Report = 31,

        [System.Runtime.Serialization.EnumMemberAttribute]
        ReportEntity = 32,

        [System.Runtime.Serialization.EnumMemberAttribute]
        ReportCategory = 33,

        [System.Runtime.Serialization.EnumMemberAttribute]
        ReportVisibility = 34,

        [System.Runtime.Serialization.EnumMemberAttribute]
        Attachment = 35,

        [System.Runtime.Serialization.EnumMemberAttribute]
        EmailTemplate = 36,

        [System.Runtime.Serialization.EnumMemberAttribute]
        ContractTemplate = 37,

        [System.Runtime.Serialization.EnumMemberAttribute]
        KbArticleTemplate = 38,

        [System.Runtime.Serialization.EnumMemberAttribute]
        MailMergeTemplate = 39,

        [System.Runtime.Serialization.EnumMemberAttribute]
        DuplicateRule = 44,

        [System.Runtime.Serialization.EnumMemberAttribute]
        DuplicateRuleCondition = 45,

        [System.Runtime.Serialization.EnumMemberAttribute]
        EntityMap = 46,

        [System.Runtime.Serialization.EnumMemberAttribute]
        AttributeMap = 47,

        [System.Runtime.Serialization.EnumMemberAttribute]
        RibbonCommand = 48,

        [System.Runtime.Serialization.EnumMemberAttribute]
        RibbonContextGroup = 49,

        [System.Runtime.Serialization.EnumMemberAttribute]
        RibbonCustomization = 50,

        [System.Runtime.Serialization.EnumMemberAttribute]
        RibbonRule = 52,

        [System.Runtime.Serialization.EnumMemberAttribute]
        RibbonTabToCommandMap = 53,

        [System.Runtime.Serialization.EnumMemberAttribute]
        RibbonDiff = 55,

        [System.Runtime.Serialization.EnumMemberAttribute]
        SavedQueryVisualization = 59,

        [System.Runtime.Serialization.EnumMemberAttribute]
        SystemForm = 60,

        [System.Runtime.Serialization.EnumMemberAttribute]
        WebResource = 61,

        [System.Runtime.Serialization.EnumMemberAttribute]
        SiteMap = 62,

        [System.Runtime.Serialization.EnumMemberAttribute]
        ConnectionRole = 63,

        [System.Runtime.Serialization.EnumMemberAttribute]
        FieldSecurityProfile = 70,

        [System.Runtime.Serialization.EnumMemberAttribute]
        FieldPermission = 71,

        [System.Runtime.Serialization.EnumMemberAttribute]
        PluginType = 90,

        [System.Runtime.Serialization.EnumMemberAttribute]
        PluginAssembly = 91,

        [System.Runtime.Serialization.EnumMemberAttribute]
        SdkMessageProcessingStep = 92,

        [System.Runtime.Serialization.EnumMemberAttribute]
        SdkMessageProcessingStepImage = 93,

        [System.Runtime.Serialization.EnumMemberAttribute]
        ServiceEndpoint = 95,
    }

    #endregion Component Types

    internal static class Program
    {
        #region Fields

        private static XrmServices _xrm;
        private static EntityMetadata[] _entityMetadata;
        private static OptionSetMetadataBase[] _optionSets;

        #endregion Fields

        private static void Main(string[] args)
        {

            var command = string.Empty;
            var solutionName = "default";
            string solutionFilename = null;
            var managed = true;
            var publish = true;
            var activate = true;
            string targetVersion = null;
            const bool systemsettings = true;
            foreach (var values in args.Select(arg => arg.Contains('=') ? arg.Split('=') : arg.Split(':')))
            {
                if (values.Length > 1 && values[1].StartsWith("\"", StringComparison.Ordinal) && values[1].EndsWith("\"", StringComparison.Ordinal))
                {
                    values[1] = values[1].Substring(1, values[1].Length - 2);
                }

                switch (values[0].ToLower(CultureInfo.CurrentCulture))
                {
                    case "activate":
                        activate = values[1] == "true" || values[1] == "yes";
                        break;
                    case "managed":
                        managed = values[1] == "true" || values[1] == "yes";
                        break;
                    case "publish":
                        publish = values[1] == "true" || values[1] == "yes";
                        break;
                    case "solution":
                    case "solutionname":
                        solutionName = values[1];
                        break;
                    case "solutionfile":
                    case "solutionfilename":
                        solutionFilename = values[1];
                        break;
                    case "systemsettings":
                        publish = values[1] == "true" || values[1] == "yes";
                        break;
                    case "version":
                        targetVersion = values[1];
                        break;
                    default:
                        if (values.Length == 1)
                        {
                            command = values[0].ToLower(CultureInfo.CurrentCulture);
                        }

                        break;
                }
            }

            if ((String.IsNullOrEmpty(command) || command == "import") && solutionFilename == null)
            {
                Console.WriteLine("Syntax :");
                Console.WriteLine("solutionmanager export [solution=<name>] [solutionfile=<filename>] [managed=yes|no] [XRM connection options]");
                Console.WriteLine("solutionmanager import solutionfile=<filename> [activate=yes|no] [publish=yes|no] [XRM connection options]");
                Console.WriteLine("solutionmanager lockdown [solution=<name>] [XRM connection options]");
                Console.WriteLine("activate     : Yes|No : On import, activate workflows & plugins. Defaults to Yes");
                Console.WriteLine("managed      : Yes|No : Export the solution as managed. Defaults to Yes");
                Console.WriteLine("publish      : Yes|No : On import, publish the solution. Defaults to Yes");
                Console.WriteLine("solution     : Name of the solution to export. Defaults to 'Default'");
                Console.WriteLine("solutionfile : Solution file to import or export to. On export defaults to '<solutionname_[managed]_versionnumber>.zip'");
                Console.WriteLine("systemsettings : Yes|No : Export the CRM System Settings as part of the solution. Defaults to Yes");
                Console.WriteLine("version      : Specifies the target version to which the file is to be imported");
                Console.WriteLine("\r\nXRM connection options can be specified via the command line or in datamapmanager.exe.config:");
                XRMConnectionDetails.DisplayCommandSyntax();
                Environment.Exit(1);
            }

            try
            {
                var connectionDetails = new XRMConnectionDetails(args);
                _xrm = new XrmServices(connectionDetails);
                _xrm.Timeout = new TimeSpan(8, 0, 0);
                _xrm.XrmServiceProxy.ServiceConfiguration.CurrentServiceEndpoint.Binding.OpenTimeout = _xrm.Timeout;
                _xrm.XrmServiceProxy.ServiceConfiguration.CurrentServiceEndpoint.Binding.ReceiveTimeout = _xrm.Timeout;
                _xrm.XrmServiceProxy.ServiceConfiguration.CurrentServiceEndpoint.Binding.SendTimeout = _xrm.Timeout;
                _xrm.XrmServiceProxy.ServiceConfiguration.CurrentServiceEndpoint.Binding.CloseTimeout = _xrm.Timeout;
                switch (command)
                {
                    case "export":
                        Export(solutionName, solutionFilename, managed, systemsettings, targetVersion, _xrm);
                        break;
                    case "import":
                        Import(solutionFilename, activate, publish, _xrm);
                        break;
                    case "lockdown":
                        LockDown(solutionName, _xrm);
                        break;
                    default:
                        Console.WriteLine("Unknown command: " + command);
                        Environment.Exit(1);
                        break;
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("System error: " + ex.Message + "\r\n" + ex.StackTrace);
                Environment.Exit(1);
            }
        }

        /// <summary>
        /// Export a XRM solution to a file.
        /// </summary>
        /// <param name="solutionName">Name of the solution to export</param>
        /// <param name="filename">Filename of the solution to export to.</param>
        /// <param name="managed">True to export a managed solution.</param>
        /// <param name="systemSettings">True to export the System Settings</param>
        /// <param name="targetVersion">To specify a particular CRM version for the export file</param>
        /// <param name="xrm">The XRM service connection.</param>
        private static void Export(string solutionName, string filename, bool managed, bool systemSettings, string targetVersion, XrmServices xrm)
        {
            string task = string.Empty;
            try
            {
                Console.WriteLine("Exporting solution " + solutionName);
                var solutionQuery = from solution in xrm.XrmContext.CreateQuery("solution")
                                    where solution.GetAttributeValue<string>("uniquename") == solutionName
                                    select new
                                    {
                                        ismanaged = (bool)solution["ismanaged"],
                                        solutionid = solution.GetAttributeValue<Guid>("solutionid"),
                                        version = (string)solution["version"]
                                    };
                var solutionInfo = solutionQuery.SingleOrDefault();
                if (solutionInfo == null)
                {
                    Console.WriteLine(solutionName + " does not exist.");
                    Environment.Exit(1);
                }

                string version = solutionInfo.version;
                if (solutionInfo.ismanaged || solutionName.Equals("Default"))
                {
                    managed = true;
                }
                else
                {
                    // Update the solution version number
                    var versionDetails = version.Split('.');
                    version = versionDetails[0] + "." + DateTime.Now.Year + "." + DateTime.Now.Month + "." + DateTime.Now.Day;
                    var solutionEntity = new Entity("solution") { Id = solutionInfo.solutionid };
                    solutionEntity["version"] = version;
                    task = "updating solution version number";
                    xrm.Update(solutionEntity);
                }

                if (String.IsNullOrEmpty(filename))
                {
                    filename = managed ? solutionName + "_managed_" + version + ".zip" : solutionName + "_" + version + ".zip";
                }

                task = "exporting solution from CRM";
                var exportSolutionRequest = new ExportSolutionRequest
                {
                    ExportAutoNumberingSettings = systemSettings,
                    ExportCalendarSettings = systemSettings,
                    ExportCustomizationSettings = systemSettings,
                    ExportEmailTrackingSettings = systemSettings,
                    ExportGeneralSettings = systemSettings,
                    ExportIsvConfig = systemSettings,
                    ExportMarketingSettings = systemSettings,
                    ExportOutlookSynchronizationSettings = systemSettings,
                    Managed = managed,
                    SolutionName = solutionName
                };
                if (targetVersion != null)
                    exportSolutionRequest.TargetVersion = targetVersion;
                var exportSolutionResponse = (ExportSolutionResponse)xrm.Execute(exportSolutionRequest);

                task = "writing solution to " + filename;
                File.WriteAllBytes(filename, exportSolutionResponse.ExportSolutionFile);
                Console.WriteLine(solutionName + " has been exported to " + filename);
            }
            catch (System.ServiceModel.FaultException<OrganizationServiceFault> ex)
            {
                Console.WriteLine("XRM Error " + task + ": " + XrmServices.GetXrmError(ex));
                Environment.Exit(1);
            }
            catch (Exception ex)
            {
                Console.WriteLine("Error " + task + ": " + ex.Message);
                Environment.Exit(1);
            }
        }

        /// <summary>
        /// Import a solution.
        /// </summary>
        /// <param name="filename">The solution file name.</param>
        /// <param name="activate">Activate workflows & plugins.</param>
        /// <param name="publish">Publish the solution components.</param>
        /// <param name="xrm">The XRM service</param>
        private static void Import(string filename, bool activate, bool publish, XrmServices xrm)
        {
            string task = string.Empty;
            Guid jobid = Guid.NewGuid();
            try
            {
                Console.WriteLine("Importing solution " + filename);
                task = "reading solution file";
                byte[] fileBytes = File.ReadAllBytes(filename);

                task = "importing solution to CRM";
                var importSolutionRequest = new ImportSolutionRequest
                {
                    CustomizationFile = fileBytes,
                    ImportJobId = jobid,
                    OverwriteUnmanagedCustomizations = true,
                    PublishWorkflows = activate
                };
                xrm.Execute(importSolutionRequest);

                XmlNode resultNode;
                string solutionName;
                XmlDocument importLog;
                for (bool first = true; ; first = false)
                {
                    var importJob = xrm.Retrieve("importjob", jobid, new ColumnSet(new[] { "data", "progress", "solutionname" }));
                    solutionName = (string)importJob["solutionname"];
                    var importData = (string)importJob["data"];
                    importLog = new XmlDocument();
                    importLog.LoadXml(importData);
                    resultNode = importLog.SelectSingleNode("//solutionManifest/result/@result");
                    if (resultNode != null) break;
                    if (first)
                    {
                        Console.WriteLine("Waiting for CRM.");
                        if (!Console.IsInputRedirected)
                            Console.WriteLine("Press q if you want to stop waiting.  Note that this won't abort the import.");
                    }

                    System.Threading.Thread.Sleep(1000);
                    if (!Console.IsInputRedirected && Console.KeyAvailable && Console.ReadKey(true).Key == ConsoleKey.Q) break;
                    Console.Write('.');
                }

                Console.WriteLine();
                if (resultNode == null)
                {
                    Console.WriteLine("Import of " + solutionName + " aborted.  Check the solution history in XrmToolbox to see whether the the import succeeded or not.");
                    Environment.Exit(1);
                }

                var result = resultNode.Value;
                var singleNode = importLog.SelectSingleNode("//solutionManifest/result/@errortext");
                if (singleNode != null)
                {
                    string errorText = singleNode.Value;
                    if (result == "failure")
                    {
                        Console.WriteLine("Import of " + solutionName + " has failed: " + errorText);
                        Environment.Exit(1);
                    }
                }

                Console.WriteLine(solutionName + " has been imported from " + filename);

                if (!publish) return;
                // Publish the solution components
                var pubRequest = new PublishAllXmlRequest();
                task = "publishing solution";
                Console.WriteLine("Publishing solution " + solutionName);
                xrm.Execute(pubRequest);
                Console.WriteLine("Finished");
            }
            catch (System.ServiceModel.FaultException<OrganizationServiceFault> ex)
            {
                Console.WriteLine("XRM Error " + task + ": " + XrmServices.GetXrmError(ex));

                // Try & get the error details
                try
                {
                    var importJob = xrm.Retrieve("importjob", jobid, new ColumnSet(new[] { "data" }));
                    var importData = (string)importJob["data"];
                    var importLog = new XmlDocument();
                    importLog.LoadXml(importData);
                    string xmlfilename = "ImportLog.xml";
                    var xmlfile = new FileInfo(xmlfilename);
                    using (var sw = xmlfile.CreateText())
                    {
                        importLog.Save(sw);
                    }

                    Console.WriteLine("Import log has been written to " + xmlfilename);
                    var selectSingleNode = importLog.SelectSingleNode("//solutionManifest/result/@errorcode");
                    if (selectSingleNode != null)
                    {
                        string errorCode = selectSingleNode.Value;
                        if (errorCode == "0x8004801D")
                        {
                            var nodes = importLog.SelectNodes("//solutionManifest/result/parameters/parameter");
                            if (nodes != null && nodes.Count >= 2)
                            {
                                var dependencies = nodes[1];
                                xmlfilename = "MissingDependencies.xml";
                                xmlfile = new FileInfo(xmlfilename);
                                var missing = new XmlDocument();
                                missing.LoadXml(dependencies.InnerText);
                                using (var sw = xmlfile.CreateText())
                                {
                                    missing.Save(sw);
                                }

                                Console.WriteLine("Missing dependencies have been written to " + xmlfilename);
                            }
                        }
                    }
                }
                catch
                {
                }

                Environment.Exit(1);
            }
            catch (Exception ex)
            {
                var msg = ex.Message;
                var innerException = ex.InnerException;
                while (innerException != null)
                {
                    msg += "\r\n" + innerException.Message;
                    innerException = innerException.InnerException;
                }

                Console.WriteLine("Error " + task + ": " + msg);
                Environment.Exit(1);
            }
        }

        /// <summary>
        /// LockDown sets all the solution components to IsCustomisable=false.
        /// </summary>
        /// <param name="solutionName">The solution name to lcok down.</param>
        /// <param name="xrm">The XRM services.</param>
        private static void LockDown(string solutionName, XrmServices xrm)
        {
            string task = string.Empty;
            try
            {
                Console.WriteLine("Locking down solution " + solutionName);
                var solutionQuery = from solution in xrm.XrmContext.CreateQuery("solution")
                                    where solution.GetAttributeValue<string>("uniquename") == solutionName
                                    select new
                                    {
                                        solutionid = solution.GetAttributeValue<Guid>("solutionid"),
                                    };
                var solutionInfo = solutionQuery.SingleOrDefault();
                if (solutionInfo == null)
                {
                    Console.WriteLine(solutionName + " does not exist.");
                    Environment.Exit(1);
                }

                GetEntityMetadata();

                // Get all the solution components
                var componentQuery = from cmp in xrm.XrmContext.CreateQuery("solutioncomponent")
                                     where cmp.GetAttributeValue<Guid>("solutionid") == solutionInfo.solutionid
                                     orderby cmp["componenttype"]
                                     select new
                                     {
                                         componenttype = cmp.GetAttributeValue<OptionSetValue>("componenttype"),
                                         objectid = cmp.GetAttributeValue<Guid>("objectid")
                                     };
                var components = componentQuery.ToList();
                int componentType = 0;
                foreach (var component in components)
                {
                    switch (component.componenttype.Value)
                    {
                        case (int)ComponentType.Entity:
                            task = "Lockdown entity";
                            if (component.componenttype.Value != componentType)
                            {
                                Console.WriteLine("  Processing Entities");
                                componentType = component.componenttype.Value;
                            }

                            LockdownEntity(component.objectid);
                            break;

                        case (int)ComponentType.ConnectionRole:
                            task = "Lockdown connection role";
                            if (component.componenttype.Value != componentType)
                            {
                                Console.WriteLine("  Processing Connection Roles");
                                componentType = component.componenttype.Value;
                            }

                            LockdownGeneric("connectionrole", component.objectid);
                            break;

                        case (int)ComponentType.OptionSet:
                            task = "Lockdown option set";
                            if (component.componenttype.Value != componentType)
                            {
                                Console.WriteLine("  Processing Option Sets");
                                componentType = component.componenttype.Value;
                            }

                            LockdownOptionSet(component.objectid);
                            break;

                        case (int)ComponentType.Report:
                            task = "Lockdown report";
                            if (component.componenttype.Value != componentType)
                            {
                                Console.WriteLine("  Processing Reports");
                                componentType = component.componenttype.Value;
                            }

                            LockdownGeneric("report", component.objectid);
                            break;

                        case (int)ComponentType.Role:
                            task = "Lockdown role";
                            if (component.componenttype.Value != componentType)
                            {
                                Console.WriteLine("  Processing Roles");
                                componentType = component.componenttype.Value;
                            }

                            LockdownGeneric("role", component.objectid);
                            break;

                        case (int)ComponentType.SdkMessageProcessingStep:
                            task = "Lockdown sdk message processing step";
                            if (component.componenttype.Value != componentType)
                            {
                                Console.WriteLine("  Processing SDK Message Processing Steps");
                                componentType = component.componenttype.Value;
                            }

                            LockdownGeneric("sdkmessageprocessingstep", component.objectid);
                            break;

                        case (int)ComponentType.SystemForm:
                            task = "Lockdown system form";
                            if (component.componenttype.Value != componentType)
                            {
                                Console.WriteLine("  Processing System Dashboards");
                                componentType = component.componenttype.Value;
                            }

                            LockdownGeneric("systemform", component.objectid);
                            break;

                        case (int)ComponentType.WebResource:
                            task = "Lockdown web resource";
                            if (component.componenttype.Value != componentType)
                            {
                                Console.WriteLine("  Processing Web Resources");
                                componentType = component.componenttype.Value;
                            }

                            LockdownGeneric("webresource", component.objectid);
                            break;
                    }
                }
            }
            catch (System.ServiceModel.FaultException<OrganizationServiceFault> ex)
            {
                Console.WriteLine("XRM Error processing " + task + ": " + XrmServices.GetXrmError(ex) + "\r\n" + ex.StackTrace);
                Environment.Exit(1);
            }
            catch (Exception ex)
            {
                Console.WriteLine("Error processing " + task + ": " + ex.Message + "\r\n" + ex.StackTrace);
                Environment.Exit(1);
            }
        }

        /// <summary>
        /// Lock down an entity.
        /// </summary>
        /// <param name="entityId">The metadataid of the entity to lockdown.</param>
        private static void LockdownEntity(Guid entityId)
        {
            var singleEntityMetadata = _entityMetadata.Single(em => em.MetadataId == entityId);
            if (singleEntityMetadata.IsManaged != null && (!singleEntityMetadata.IsCustomizable.Value || singleEntityMetadata.IsManaged.Value)) return;
            singleEntityMetadata.IsCustomizable.Value = false;
            var updateEntityRequest = new UpdateEntityRequest
            {
                Entity = singleEntityMetadata
            };
            Console.WriteLine("    Locking down entity: " + singleEntityMetadata.LogicalName);
            _xrm.Execute(updateEntityRequest);
        }

        /// <summary>
        /// Lock down an OptionSet
        /// </summary>
        /// <param name="optionSetId">The ID of the option set to lock down.</param>
        private static void LockdownOptionSet(Guid optionSetId)
        {
            if (_optionSets == null)
            {
                var raoRequest = new RetrieveAllOptionSetsRequest
                {
                    RetrieveAsIfPublished = false
                };
                var raoResponse = (RetrieveAllOptionSetsResponse)_xrm.Execute(raoRequest);
                _optionSets = raoResponse.OptionSetMetadata;
            }

            // Get the option set
            var optionSet = _optionSets.Single(os => os.MetadataId == optionSetId);
            if (optionSet.IsGlobal == null ||
                (!optionSet.IsGlobal.Value || !optionSet.IsCustomizable.Value || !optionSet.IsCustomizable.CanBeChanged))
                return;
            optionSet.IsCustomizable.Value = false;
            var uosRequest = new UpdateOptionSetRequest
            {
                OptionSet = optionSet
            };
            Console.WriteLine("    Locking down option set: " + optionSet.Name);
            _xrm.Execute(uosRequest);
        }

        /// <summary>
        /// Lockdown a component that is defined as a XRM entity.
        /// </summary>
        /// <param name="entityName">Entity name to lock down.</param>
        /// <param name="entityId">Id of the entity.</param>
        private static void LockdownGeneric(string entityName, Guid entityId)
        {
            var entity = _xrm.Retrieve(entityName, entityId, new ColumnSet(true));
            var customizable = entity.GetAttributeValue<BooleanManagedProperty>("iscustomizable");
            if (!customizable.CanBeChanged || !customizable.Value) return;
            var updateEntity = new Entity(entityName)
            {
                Id = entityId
            };
            customizable.Value = false;
            updateEntity["iscustomizable"] = customizable;
            Console.WriteLine("   Locking down " + entityName + ": " + entity["name"]);
            _xrm.Update(updateEntity);
        }

        /// <summary>
        /// Get the Entity metadata.
        /// </summary>
        private static void GetEntityMetadata()
        {
            Console.WriteLine("  Retrieving Entity metadata.");
            var raeRequest = new RetrieveAllEntitiesRequest
            {
                EntityFilters = EntityFilters.Entity,
                RetrieveAsIfPublished = false
            };
            var raeResponse = (RetrieveAllEntitiesResponse)_xrm.Execute(raeRequest);
            _entityMetadata = raeResponse.EntityMetadata;
        }
    }
}

Developers need to do below the configuration in visual studio(Run the command prompt from inside Visual Studio) to execute the SolutionExportDev.cmd

To make the tool available, add it to the external tools list. Here are the steps:

  1. Open Visual Studio.
  2. Select the Tools menu, and then choose External Tools.
  • On the External Tools dialog box, choose the Add button. A new entry appears.
  • Enter a Title for your new menu item such as Command Prompt.
  • In the Command field, specify the file you want to launch, such as SolutionExportDev.cmd
  • In the Arguments field, specify the organization name, CRM username, password, server name, and domain name
  • Choose a value for the Initial directory field, such as Project Directory.

Example:

Title – Name of the external tools to run

Command – Provide the physical path to the command file

Arguments – SolutionExportDev.cmd accepts 4 arguments

  • Organization Name
    1. Username
    2. Password
    3. CRM Server name
    4. Domain

Initial directory – Provide the directory path of the SolutionExportDev.cmd file.

Use Output Window – Enable

Prompt for Arguments ­– Disable

Close on exit–Enable

  • Choose the OK button.

The new menu item is added, and you can access the command prompt from the Tools menu as follows:

To run this, please click on ExportCrmSolution. It will export the solution from source instance and unpack the solutions into the folders in BuildAutomation. Solutions.

Once the above step is done, the developer now has some manual steps to perform depending on the files/changes that were extracted from the solution:

  1. Include any new file under the extracted folder Solutions\BuildAutomation (extracted) into the project- BuildAutomation. Solutions (so that they can be checked-in).
    NOTE: When doing the check-in, be sure to check the ‘Excluded Changes’ if additional files were detected and promote them to the ‘Included Changes’ as needed.
  2. For any .xaml file (under Solutions\BuildAutomation\Workflows or Solutions\BuildAutomation \Entities\xxxx\Formulas), make sure that Build Action is ‘None’ (not ‘Page’ or ‘XamlAppDef’). If this step is skipped, you may get one of this compilation error (s):
    – Project file must include the .NET Framework assembly ‘WindowsBase, PresentationCore, PresentationFramework’ in the reference list.
    – Assembly “System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a’ cannot be resolved”.
  • Ensure that DLL files under Solutions\..\PluginAssemblies are added to source control.
  • Undo all pending changes that do not specifically apply to your changes.
  • Associate the check-in with a work item(s), write a comment and check-in all required changes.

Note: Check-in to TFS will auto trigger the deployment process.

In my next blog, we will see how to pack the solution and deploy it in the Target instance using VSTS Build and Release definition.

If you are interested in this topic and would like to do some further self-study I encourage you to check out my blog on this.

4 thoughts on “CI/CD & Test Automation for Dynamics 365 in Azure DevOps/VSTS – Part 1

  1. Pingback: Continuous Integration, Deployment & Test Automation for Dynamics 365 CE – Part 2 – Microsoft Dynamics CRM/365 Blog

  2. Pingback: Continuous Integration, Deployment & Test Automation for Dynamics 365 CE in Azure DevOps/VSTS – Part 1 - Microsoft Dynamics CRM Community

  3. Pingback: Continuous Integration, Deployment & Test Automation for Dynamics 365 for Customer Engagement | Let Us Discuss

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s