Integration of Dynamics CRM 2011 solutions with TFS

December 28, 2012

Some weeks ago I was asked for a proof of concept to design a TFS 2010 solution to fully (not less than 100%) automate a complex Dynamics CRM 2011 deployment for various environments (dev / test / staging / production). Many different components were involved: the CRM solution itself, but also web applications, database objects, reports (SSRS), transformations, …

Dynamics CRM

It has been an interesting journey so far and along the way I got to know (a bit) how Dynamics CRM 2011 is working. Not to my surprise, I realized that it’s quite hard to push all source related items to TFS and to force ALL changes/updates to a CRM environment from a version controlled solution in TFS. Many things in the CRM environment are easily modified by the development team via the CRM UI web interface and as result, directly stored in the CRM database(s). So, the POC also required me to think about enforcing best practices for the CRM development team to avoid inconsistencies in the global deployment solution and I definitly wanted to end up with a build-once;deploy-many solution.

Anyway, I won’t talk about the entire scope of the POC, but I want to highlight the approach I took for automating the export & extract operation from the development CRM 2011 instance via a TFS build definition. The goal here was to automatically capture the daily changes which were published to deployed CRM development solutions.

EXPORT
The MSCRM 2011 Toolkit contains a Solution Export command line utility which enabled me to export one or multiple CRM solutions from an existing Solutions Export Profile into a single compressed solution file (.zip).

EXTRACT
The compressed solution file (zip-format) is of course not ideal to track the individual changes and to bind it to a version control repository. Luckily, with the latest release of the Dynamics CRM 2011 SDK, a new tool (SolutionPackager) was added to extract the different components into individual files.

The SolutionPackager tool, available in the Microsoft Dynamics CRM 2011 Update Rollup 10 version of the Microsoft Dynamics CRM SDK download, resolves the problem of source code control and team development of solution files. The tool identifies individual components in the compressed solution file and extracts them out to individual files. The tool can also re-create a solution file by packing the files that had been previously extracted. This enables multiple people to work independently on a single solution and extract their changes into a common location. Because each component in the solution file is broken into multiple files, it becomes possible to merge customizations without overwriting prior changes. A secondary use of the SolutionPackager tool is that it can be invoked from an automated build process to generate a compressed solution file from previously extracted component files without needing an active Microsoft Dynamics CRM server.

TFS 2010

So, these tools opened the door for me to work out a custom build process (workflow) in TFS 2010 with the following sequential activities:

  • Export CRM solution from dev environment (MSCRM 2011 Toolkit)
  • Prepare TFS workspace before extract of solution file [Get Latest + Check-Out]
  • Extract compressed solution file into TFS workspace (SolutionPackager)
  • Scan TFS workspace for changes/additions/deletions
  • Check-In pending changes of the TFS workspace as a single changeset

The scan of the TFS workspaces – to end up with all differences [changes/additions/deletions] – was a bit more complex than expected because I needed to use several TFS API Workspace calls like PendEdit, PendAdd, PendDelete, … I also made use of the EvaluateCheckin2 method to detect potential conflicts and to perform proper exception handling.

This process allows the development team to easily follow-up the incremental changes (via TFS changesets) which were applied to the dev CRM environment. Note that the SolutionPackager tool is also able to generate a compressed solution file from the individual component files.


Version Control with Team Foundation Server 2010

January 11, 2011

On February 15, I will do a Microsoft Live Webcast on “What you should know about Version Control in Team Foundation Server 2010”.

Unlike Visual SourceSafe (VSS), which relied on a file-based storage mechanism, Team Foundation version control stores all artefacts, as well as a record of all changes and current check-outs in a SQL Server database which makes it highly reliable and scalable. It supports features such as multiple simultaneous check-outs, conflict resolution, shelving and unshelving, branching and merging, and the ability to set security levels on any level of a source tree, alongside the most visible features of document versioning, locking, rollback, and atomic commits. The source control mechanism integrates with TFS Work Item Management as well. TFS administrators can enforce check-in policies that require specific requirements to have passed and individual versions of files can be assigned labels. This session is targeted towards developers who want to know all the details about the new version control features in Team Foundation Server 2010.

Read more.

Update [May 4, 2011]: recording uploaded to Channel9

Watch recording


The Gated Check-in build in TFS2010

April 18, 2010

Everybody should be already familiar with Continuous Integration or should I say Continuous Building? Automatically building a development codeline after a check-in is often not immediately followed by an integration action towards a main branch. I picked up the term Continuous Building in this article of Martin Fowler.

Apart from the fact how this “build automation” should be called, there are many reasons why you should enforce this behavior on different branch types for your applications. The ultimate goal is to improve the quality of the software application and to reduce the time to release the application in production. By setting up early validation (compilation, automatic testing + other quality gates) through “build automation” you will at least be notified as soon as possible of all kinds of validation errors (= quality check) and you will have a chance to fix them before other team members will be impacted by pulling a get latest on the repository.

Automatically firing a validation build after a check-in will in the end not prevent broken builds and that’s where the Gated Check-in Build will come into play with Team Foundation Server 2010.

The Gated Check-in Build in TFS2010 will prevent broken builds by not automatically committing your pending changes to the repository, but the system will instead create a separate shelveset that will be picked up by the Gated Check-in Build. The build itself will finally decide if the pending changes need to be committed to the repository based on the applied quality gates.

Gated Check-In Build process

The picture above describes the full process of a Gated Check-In build.

How to setup a Gated Check-in build?

The Trigger Tab in the Build Definition window has now an extra option for selecting Gated Check-in.

GatedCheckIn2

At the moment a check-in is attempted by a developer in the branch where the Gated Check-in build is active, the developer will be faced with a dialog box.

GatedCheckIn3

Cancelling this window will not kick off the build, but will also not commit your pending changes to the repository. If you really want to overrule this build with committing your changes directly to the repository, you may select the 2nd checkbox to bypass the validation build (not recommended). By default your pending changes will reside in your local workspace (first checkbox). In the situation where you immediately want to start with new changes – not relying on previous changes – it might be appropriate to uncheck the first option.

In the ideal situation, the build will complete without any validation errors and will eventually commit the changes to the repository. This will also lead to a Gated Check-in notification for the original committer via the Team Build Notification tool.

GatedCheckIn5

GatedCheckIn4

If you had previously chosen to preserve the changes locally (default), you may have noticed that the files you were working on were still checked out during the build … and in fact after a successful build these changes do not reflect the as-is situation anymore of the repository. With the above window you get the option to immediately reconcile your workspace with the up-to-date repository. So, clicking the “Reconcile …” button will give you the opportunity to select the desired files to force an undo in your local workspace and to pickup the changes that were committed by the Gated Check-in build for these files.

Another way to reconcile your workspace (if you for example ignored this window or when the build notification is way too slow) is by right-clicking the completed Gated Check-in Build in the Build Explorer and selecting the option to reconcile your workspace.

GatedCheckIn6  

If you did not choose to preserve the changes locally, there won’t be any changes to reconcile after the Gated Check-in build, even if you forced the reconciliation.

GatedCheckIn8

The Gated Check-in build may also be kicked off manually where you may need to create a shelveset or where you may point to an existing shelveset.

GatedCheckIn9

A last thing to note is that the comment that was originally supplied to the changeset by the developer will be suffixed with the NoCICheckinComment variable (default = ***NO_CI***) to prevent that another continuous integration build will be fired after the final check-in done by the Gated Check-in build.

GatedCheckIn7

Summary

What meant to be a small post on the Gated Check-in feature in Team Foundation Server 2010 ended up in a more detailed explanation of how it works and how you can work with it in the Visual Studio IDE. Remember that you should setup the most appropriate build types according to your specific branches. Not all branches may need a Gated Check-in build. Only configure this for branches that should never have a broken build. A Gated Check-in build may for example validate a big merge operation from a development branch to a stable main branch.


Techdays Belgium 2010 – Session details

April 1, 2010

This year I presented a session at Techdays Belgium on Branching & Merging with Team Foundation Server 2010.

The session slides can be downloaded in the download section of this blog.

Watch recorded video (1 hour and 10 minutes) at Channel 9.

A demo on Branching & Merging with TFS2010 was the major part during the presentation and I did practically cover everything I wanted to share with the audience: Branch metadata, Fine-grained security, Branching Visualization, Tracking individual changesets across branches, forward/reverse integration … except for one little important merge action that I forgot to show!

After the creation of my dev branches (from main) I also renamed the solution in those dev branches with an additional suffix to avoid confusion while loading different solutions into Visual Studio 2010. This was done in changeset 121 and I renamed the solution from WebsiteSparkles to WebsiteSparkles_dev1.

DemoTechdays1

Afterwards I did some code changes in the dev branches and pushed some explicit changesets back (Reverse Integration) to the main branch, using the cherry pick option in the merge wizard to avoid merging also the solution rename.

As a result changeset 121 will always remain a merge candidate in the Source Control Merge Wizard.

DemoTechdays2

In some cases you really want to merge changes back to main on the latest version of the development branch without cherry-picking all required changesets. To be able to do that, you need to get rid of changeset 121 as a merge candidate.

This can only be done through the command-line with the tf merge /discard command.

DemoTechdays3

This discard command will make sure that changeset 121 will not be a merge candidate anymore. Note that you still need to commit this action to the repository after executing the command. The discard command will only update your local workspace but won’t do an automatic check-in.

DemoTechdays4

Next time you will run the merge wizard and look for merge candidates, changeset 121 won’t be listed anymore and you may merge from a latest version of this development branch for upcoming changes.

Providing this discard command from within the source control merge wizard would be a very nice addition!


Deleted Team Projects still visible in Source Control Explorer

November 25, 2009

Today I bumped into a solution for deleting deleted Team Projects (yip, that’s right: deleting Team Projects that should have been already deleted). A while ago we noticed that Team Projects in TFS2008 SP1 that were deleted with TFSDeleteProject.exe were still visible in the Source Control Explorer and re-running TFSDeleteProject.exe for the same Team Projects resulted in an error message that the Team Project could not be found on the Team Foundation Server.

But due to this entry in the TFS forum I was finally able to remove the Team Projects from Source Control Explorer as well.

Solution:

  • Create new Team Project with exactly the same name as the Team Project that wasn’t completely deleted and specify the option to not create the version control folder.
  • After the Team Project Creation wizard finishes, you need to run TFSDeleteProject.exe for the newly created Team Project.

No sign anymore of the deleted Team Projects in Version Control! Sounds logical?!


TFS2010: Visualization of merging partial changes in changeset

November 11, 2009

Yesterday (during the Source Code Management session of Brian Harry at TechEd 2009 in Berlin), I noticed for the first time that merging partial changes in a changeset is also clearly visible afterwards. I just found some time between sessions to test it out …

In the screenshot above you may notice that changeset 9 (dev branch) is merged to the main branch. But look at the yellow color of the bar after the merge and the tooltip that is shown when I hover over changeset 11. Changeset 9 consisted of 2 different files that were modified while the merge operation took only 1 modified file in changeset 11. Valuable information!


Branching in TFS2010: atomic operation without pending changes

November 10, 2009

Currently in TFS2008 when you create a new branch in your version control structure, all files are copied to your local worspace and marked as pending changes. This means that a branch in TFS2008 is not seen as an atomic operation. You were required to check-in the local changes before the branch got created on the server. If your branch would consist of a huge set of files, branching could become a time-consuming operation and the system would not prohibit you to do some modifications to the pending changes.

In TFS2010, a branch is now seen as an atomic operation: it doesn’t copy all files into your local workspace and the branch action is immediately checked-in – no option anymore to do some local modifications. The child branch will always be an exact copy of the selected parent branch folder. You will get a messagebox that will notify you of this action when you hit the branch button …

You will need to request a get latest on the newly created child branch to have a local copy of the sources.

Some simple changes can make a big difference!


Follow

Get every new post delivered to your Inbox.