Update appsettings.json at deploy time with VSTS Release Management

July 27, 2017

Lately I have to deal more with .NET Core Web Applications to setup build and release definitions in VSTS. What always comes up is how to deal with specific application settings which must me updated for a specific environment.

I have always been a big advocate of making a clear separation between build and release. The build should simply generate a generic package while the release should pick up the package and deploy it to any possible environment. At deployment time the specific enviroment values should be injected. Web Deploy has been the obvious tool in the past to make this happen with the capability to update the generated setparameters.xml file in a deployment action which injected the environment values into the web.config file.

Now with .NET Core and the typical appsettings.json file, it has become really easy in VSTS to inject custom values into the appsettings.json file.

Example of appsettings.json file in my solution:

AppSettings

Imagine that you would want to replace the values for the different settings (as from line 8). Note that it will also be possible to replace the values in the “Administrators” array.

First, you will need to create a build in VSTS which produces the deployment zip package (via dotnet publish command).

BuildOutput

The VSTS release definition will link to the build output and you can use the built-in release task “Azure App Service Deploy” to deploy the build output to an Azure App Service.

ReleaseTask

The “File Transformation” section in the release task offers the possibility to define the JSON variable substitution. You will need to provide the file name from the root and the environment values (pay attention to the format of the variable names) can be set for the “DEV” environment.

Variables

Doing a file lookup from the console in the Azure Portal after deployment shows the result of the appsettings file.

image

Simple solutions are always the best solutions!

Advertisements

Refresh git remote branches in Visual Studio

January 17, 2017

Visual Studio doesn’t always refresh the git remote/published branches in the Branches View.

My solution to force a sync in Visual Studio is calling the git remote “prune” command (https://git-scm.com/docs/git-remote). This command will immediately detect new remote branches or remove the “stale” branches. Instant update in Visual Studio.

SNAGHTML1465e1b2


Migrate inline images to VSTS

September 14, 2016

Lately I have been planning a number of migrations to move small/big companies from TFS on-premises to Visual Studio Team Services (VSTS).

There are a number of options to migrate data from TFS to VSTS, but option 3 [high-fidelity database migration] is unfortunately not yet available. So, most of the time I still use custom/third-party tooling to perform the migration which is not always straightforward and may be very time-consuming.

One serious issue that popped up in a migration towards VSTS (using the TFS Integration Tools), was that the inline images in the Description field (or other html fields) of migrated work items were not properly migrated. The inline images are actually still referring to the old TFS on-premises environment because the html value of the Description field contains an <img> element with a source set to http://<tfs-on-prem>:8080/tfs/<tpc>/workitemtracking/v1.0/attachfilehandler.ashx?filenameguid=<guid>&filename=<filename>.png.

As a result, all inline images are only stored in the old TFS environment and they have not been uploaded to VSTS. The html value of the Description field has been migrated as-is. Initially you might not notice this after the migration of the selected work items because as long as the old TFS environment is still available, the inline image will be displayed. But what if the old TFS environment has been archived/destroyed?

To correct this and to upload the original images to VSTS, I have written some code (TFS API) to loop over the VSTS work items to detect image links to the old TFS environment. Using the source link of the original image, I can download the image to my local disk and upload it as an attachment to the VSTS work item. Finally, I’m replacing the original image source link with the new VSTS image link. Good to know is that once in-line images are detected inside the html field, those images are stored on the server and the temporary image file attachments may be deleted as well.

 


Migrating TFVC (TFS on-premises) to Git (VSTS)

June 9, 2016

In the last couple of months I do get more requests to move TFVC version control history to a git repository in Visual Studio Team Services (VSTS). The migration from TFVC to TFVC is at the moment possible via the TFS Integration Tools and is not that straightforward to accomplish. Migrating to a git repository is much simpler and is certainly the way to go if you were already planning to adopt git in the future. The migration can be done via Git-TF which is a set of cross-platform command-line tools that facilitate sharing of changes between Team Foundation Server, Visual Studio Team Services and Git.

What do you need to get started?

  • Download git via https://git-scm.com/downloads
  • Download and extract Git-Tf to your computer
  • Add the extracted git-tf path to the system environment variable “path”
  • Create a new “git” Team Project in VSTS

Migration Steps:

  • Open a command-line prompt and navigate to a directory where you want to host the local git repository
  • Call git-tf clone to push all TFS changeset info from TFVC to a new local git repo. The first argument is the Team Project Collection url. You pass the TF version control path to the exact branch in the second argument and you end the command with the “deep” flag to ensure that the full history of the branch is moved into separate commits in the git repo. Pass your credentials to connect to TFS and execute the command.

git-tf-clone1

  • Once you have a local git repository it’s easy to push it towards an empty central VSTS git repository. First use the git remote add command to link your local git repo to the remote “origin” and afterwards you can push all changes via git push.

git-tf-clone2

    Navigate to the Code Hub in your VSTS Team Project and you should see all code history inside the git repo. What’s a big plus is that the original changeset date/time stamps are now part of the git commit info.


    Work Item Query via TFS API and the dayPrecision parameter

    March 7, 2016

    By default TFS doesn’t pay attention to the time part in work item queries when comparing datetime values. If you want to launch a query and you need to take into account the exact timestamp, you must switch off the dayPrecision parameter in the Query constructor.

    dayPrecision

    using the dayPrecision parameter in the Query constructor

    MSDN documentation: https://msdn.microsoft.com/en-us/library/bb133075(v=vs.120).aspx

    Mystery resolved!

     


    Global .NET Versioning Strategy – AssemblyInformationalVersion

    August 24, 2015

    Ever heard of a third (optional) versioning attribute in the AssemblyInfo files: AssemblyInformationalVersion. No? Please read!

    Without a methodical (assembly) version numbering strategy, the ability to determine what changes were included in which version is lost. In my opinion, you always need to know exactly which source files went into which build and which version of the sofware is currently deployed in a specific environment. A random version numbering system creates confusion and will soon or later cause deployment risks. It will become a nightmare to fetch the exact source files to reproduce a bug from production.

    All versioning of .NET assemblies that use the common language runtime is done at the assembly level. The specific version of an assembly and the versions of dependent assemblies are recorded in the assembly’s manifest. The default version policy for the runtime is that applications run only with the versions they were built and tested with, unless overridden by explicit version policy in configuration files.

    Each .NET project has an AssemblyInfo file which contains an AssemblyVersion attribute and an AssemblyFileVersion attribute.

    • AssemblyVersion: this is the version number used by the .NET framework during build and at runtime to locate, link and load the assemblies. When you add a reference to any assembly in your project, it is this version number which gets embedded. At runtime, the CLR looks for assembly with this version number to load. But remember this version is used along with name, public key token and culture information only if the assemblies are strong-named signed. If assemblies are not strong-named signed, only file names are used for loading.
    • AssemblyFileVersion: This is the version number given to a file as in file system. It is displayed by Windows Explorer. It’s never used by .NET framework or runtime for referencing.

    But what about this difference between AssemblyVersion and AssemblyFileVersion? Many times, I see that the same version is applied to both attributes … but why are these two (different) attributes provided by the .NET Framework? The AssemblyVersion should be the public version of an entire software application, while the AssemblyFileVersion is more the version of a specific component which may only be a small part of the entire application. The AssemblyFileVersion is the best place to put extra build version information which can be important for patching individual components of a software application.

    Please follow the Semantic Versioning recommendations to dictate how the AssemblyVersion should be assigned and incremented. For the AssemblyFileVersion, I tend to include specific build information. Often, you will need to build (and test) a number of time a specific SemVer version of your software.

    For example: release 1 of a software application could have the AssemblyVersion set to 1.0.0 (all components), while the AssemblyFileVersion of the individual components could be set to 1.0.15234.2 which refers to a unique build number of the build system and is linked to a particular date and a revision: “15” = year 2015; “234” = day number in 2015; “2” = second build processed that day. This also allows to later patch individual components in production with a similar AssemblyVersion (1.0.0), but a different AssemblyFileVersion (1.0.15235.1).

    So, let’s try to apply this to a test project in Visual Studio and see the assembly details after building the project …

    Versioning-1

    Now you should be confused! Why does the Product Version display the AssemblyFileVersion and where’s the AssemblyVersion? The problem here is that a new Visual Studio project doesn’t include a third version attribute AssemblyInformationalVersion which is intended to represent the public version of your entire software application. Note that the CLR doesn’t care about this third (optional) version attribute. In short, the same Semantic Versioning rules of AssemblyVersion should be applied to AssemblyInformationalVersion.

    Versioning-2

    Aha! This looks much better right? Now it’s also easy to extract this metadata from your deployed assemblies and this information can be nicely listed in the about box of your software. The only issue with this approach is that the AssemblyFileVersion doesn’t include the “patch” number (Semantic Versioning) of the AssemblyVersion, but this can be ignored with the fact that the AssemblyFileVersion will be unique and can be linked to a unique build run in the build system. This way of working is my personal interpretation of how versioning can be properly applied in complex software applications and doesn’t reflect official guidelines from Microsoft. My goal here is to make software developers aware of the potential risks of not having a clear versioning strategy.

    Now, forget about manually setting version information in the AssemblyInfo files and never ever release software from a local Visual Studio build. In a streamlined build process, generating unique build and version numbers are centrally coordinated. For effective troubleshooting and traceability, it’s imperative that the generated assemblies are stamped with a unique identifier that can be easily traced back to a system build number.

    In a next post I will talk about how you can achieve this global .NET versioning strategy with the new build system in TFS 2015.


    Split test runs for TFS Build and inspect test results

    September 22, 2014

    As a consultant, many times I have to deal with custom requests which cannot be handled in TFS out-of-the-box. Many customizations end up to become valuable for other customers as well. Unfortunately I don’t always find the time to write about it and to work out a more generic solution which could help other people.

    But recently I got an interesting question to intervene during the test run on the TFS Build Server because a complete test run took way too much time. The solution which was built on the server consisted of a big set of Unit Tests and a big set of Integration Tests. The Integration Tests required a deployment of a SQL Server database with some reference data. All tests were run at the same time and this caused builds to run for a long time, even if one of the Unit Tests failed at the beginning of the test run. The test run only completes (success/failure) after running ALL tests.

    So, the goal was to quickly detect when a test fails (= fail early!) and to have the possibility to stop the build process immediately after the first test failure (=stop/fail build at the point one of the tests fails). The customer didn’t see any added value to run the remaining tests, knowing that already one test failed. Instead of waiting 30’ or longer for the full test results, the developers could already start fixing the first test failure and stopping the build would also decrease the load on the build server and test environment. We also agreed to only deploy the database when all Unit Tests succeeded.

    How to separate the Integration Tests from the Unit Tests?

    image

    My sample solution above contains 2 separate projects/assemblies to host the Unit Tests and the Integration Tests. During the configuration of a Build Definition, you can easily define 2 separate test runs.

    image

    The first test run definition will only fetch the Unit Tests, while the second test run definition will look for the Integration Tests. Note that I specified a specific name for the test run definition. I will use this name later to filter the test run definitions. Creating multiple test run definitions is a flexible and easy way to split your big test run in multiple smaller test runs.

    How to filter the test run definitions before the execution of a Test Run?

    Time to customize the build process a bit so that first only the Unit Tests can be run before deciding to proceed with a database deployment and the run of the Integration Tests.

    image

    Instead of running the default VS Test Runner activity which would run all test run definitions (“AutomatedTests”), you need to filter for the Unit Tests. This can be done by modifying the TestSpec parameter for the RunAgileTestRunner activity. A where clause is added to the AutomatedTests value to search only for the “UnitTests” test run definition(s).

    image

    Result => only the Unit Tests will be executed by the VS Test Runner.

    After this test run we can check the TestStatus of the build to stop the build or (in case of no test failures) to continue with a database deployment and the run of the Integration Tests.

    In the ForEach activity after the database deployment operation I added a TestSpec filter for the AutomatedTests to only fetch the “IntegrationTests”.

    image

    The sequence in the ForEach activity will then call again the VS Test Runner and check for test failures to potentially stop the build in case of failure in the Integration Tests.

    The more fine-grained you define your Integration Tests (= different test run definitions, making use of the test assembly file specification or the test case filter), the sooner you can inspect the test results and fail the build without running the other set(s) of Integration Tests.

    Inspect Test Results during a Test Run (no filters)?

    In the beginning, I started looking into options to inspect the test results during the ongoing one-and-only test run (no different test run definitions / no requirement for filters). I quickly stumbled on this MSDN page to learn more about the VSTest.Console.exe command-line options. By adding the switch /Logger:trx it’s possible to drop the test results into a Visual Studio Test Results File (.trx), but the problem is that the .trx file is only persisted to disk once the test run finishes. I didn’t find a way to get to the test results while the test run was still executing.

    To stop the build in the customized build process template, I did throw an exception which is sufficient to stop the build process.

    You can download the build process template which I used to write this blog entry. It’s far from complete and not fully tested, but it could help you to understand the filtering of the test run definitions.