Duration of a Full Backup Job in TFS

April 24, 2017

In order to properly plan a production migration of TFS I often also check the duration of a full backup. The easiest way to see the evolution of the latest Full Backups is to run a simple SQL Query in the TFS_Configuration database …

TFSFullBackupDuration

SELECT [QueueTime], [StartTime], [EndTime], [Result], [ResultMessage], CONVERT(TIME,[EndTime] – [StartTime]) as Duration  FROM [Tfs_Configuration].[dbo].[tbl_JobHistory] jh JOIN [Tfs_Configuration].[dbo].[tbl_JobDefinition] jd on jh.JobId = jd.JobId WHERE jd.JobName = ‘Full Backup Job’ order by HistoryId desc


Upgrade TFS Team Project features

April 20, 2017

When upgrading TFS, the existing Team Projects won’t automatically adopt the new features of the new TFS version. Some of the new features might require some updates to the Team Project. Note that this will only be required for TFS Team Projects … VSTS Team Projects are automatically updated with each service upgrade.

You can perform this update yourself via the Configure Features wizard. If the Configure Features link is visible for your Team Project, it means that the Team Project requires an update. Otherwise, the new features are already enabled.

alm_cfw_configfeatures

This might work if you don’t have a lot of Team Project Collections and Team Projects. During a recent upgrade to TFS 2017 Update 1 at a customer, I was confronted with 31 Team Project Collections and in total a bit more than 400 Team Projects. No way I was going to hit the configure features link 400 times …

I remember having done this already programmatically in the past (https://www.visualstudio.com/en-us/docs/work/customize/configure-features-after-upgrade#program-updates), but the issue now was that there wasn’t a ready-to-use solution for TFS 2017 Update 1. So, I used some tips & tricks from https://www.visualstudio.com/en-us/docs/work/customize/configure-features-after-upgrade#program-updates and also the Features4tfs CodePlex solution was a good starting point. I wanted to have a scenario where it’s possible to scan a complete TFS 2017 environment with all on-line Team Project Collections and all available Team Projects.

As a result, you can find my solution in Github: https://github.com/pietergheysens/TFSUpgradeTeamProjectFeatures. Because it worked for me with TFS 2017 (Update 1), it doesn’t mean it will work for you. Please test it first during a trial-upgrade and see if it helps you to upgrade your Team Projects in one go.


Visual Studio 2017 Launch event

March 15, 2017

Last week on March 7, Microsoft officially launched Visual Studio 2017 and the Visual Studio User Group (VISUG) in Belgium organized a livestream event to watch the keynote together at the local Microsoft office in Belgium.

After the keynote, I did an extra presentation to cover the evolution of VSTS & TFS in the last couple of months.

Ever since the VSTS Product Team has started working in 3 weeks sprints to deliver new features to the product, it has been a real eye-opener to witness how fast the product is evolving and how many new features has been introduced since the beginning of Team Foundation Service, Visual Studio Online and now the current name of the product: Visual Studio Team Services (VSTS). In this demo-heavy session we will have a quick look at some of the new interesting features that were added in the last couple of months.

My slides are available on SlideShare.


TFS Production upgrades: stay calm and stick to the plan!

March 4, 2017

When planning a big migration upgrade to TFS 2017 (from TFS 2013) on new hardware, the exact planning of all actions can be very important to make sure the downtime of the TFS environment can be as short as possible and there’s at least some buffer to fix unexpected issues. That’s why I always try to perform production migrations in a week-end and that’s why you should always run a trial migration to have an idea about the total duration.

For this specific migration I’m doing this week-end, TFS 2017 is only an intermediate step because the customer also wants to migrate to VSTS from the TFS 2017 environment. I managed to do this without any issues during a trial run.

So, the plan for the production migration was: bringing the TFS environment offline on Friday evening and already launching the TFS 2017 upgrade wizard on Friday evening to make sure the long upgrade process can continue to run during the night. During the trial upgrade, this process took about 4 hours.

Unfortunately when logging back in on Saturday morning, I noticed the upgrade process failed after more than 3 hours (step 1523 of 1621) due to error TF30042: The log file for the database is full.

UpgradeError

The dedicated log disk on the server was indeed full. Seeing this error might freak you out because first you will believe that the complete upgrade failed and you need to start all over again. This might jeopardize the full plan to have a working VSTS environment on Monday morning.

This is for sure a moment to stay calm and to properly assess the situation and read all text which is available for you in the log file and also have a good look at the warning message in the TFS Upgrade wizard:

One or more project collections failed to upgrade … Start the Administration Console and navigate to the Team Project Collections node to attempt retrying the upgrade for each failed collection.

No need to start all over again! Fix the error which can be found in the error log and try to resume the upgrade process. In my situation I had to clean up the dedicated log file disk before rerunning the job from the TFS Administration Console.

ResumeUpgradeProcess

And indeed, the upgrade process resumed from step 1523 …

I only lost a bit of processing time, but still ok to finish the complete upgrade process before Monday morning …

Having done about 50 TFS upgrades in the last couple of years, I never had to cancel a production upgrade. I always delivered the new environment on time. Of course, there were times were unexpected issues came up or where I needed to perform some aftercare when the new environment was already up-and-running.

Rule #1: always have a backup plan in case of a hard failure

Rule #2: stay calm and properly assess the situation

Rule #3: call help before doing crazy stuff in a production environment


Refresh git remote branches in Visual Studio

January 17, 2017

Visual Studio doesn’t always refresh the git remote/published branches in the Branches View.

My solution to force a sync in Visual Studio is calling the git remote “prune” command (https://git-scm.com/docs/git-remote). This command will immediately detect new remote branches or remove the “stale” branches. Instant update in Visual Studio.

SNAGHTML1465e1b2


Migrate inline images to VSTS

September 14, 2016

Lately I have been planning a number of migrations to move small/big companies from TFS on-premises to Visual Studio Team Services (VSTS).

There are a number of options to migrate data from TFS to VSTS, but option 3 [high-fidelity database migration] is unfortunately not yet available. So, most of the time I still use custom/third-party tooling to perform the migration which is not always straightforward and may be very time-consuming.

One serious issue that popped up in a migration towards VSTS (using the TFS Integration Tools), was that the inline images in the Description field (or other html fields) of migrated work items were not properly migrated. The inline images are actually still referring to the old TFS on-premises environment because the html value of the Description field contains an <img> element with a source set to http://<tfs-on-prem>:8080/tfs/<tpc>/workitemtracking/v1.0/attachfilehandler.ashx?filenameguid=<guid>&filename=<filename>.png.

As a result, all inline images are only stored in the old TFS environment and they have not been uploaded to VSTS. The html value of the Description field has been migrated as-is. Initially you might not notice this after the migration of the selected work items because as long as the old TFS environment is still available, the inline image will be displayed. But what if the old TFS environment has been archived/destroyed?

To correct this and to upload the original images to VSTS, I have written some code (TFS API) to loop over the VSTS work items to detect image links to the old TFS environment. Using the source link of the original image, I can download the image to my local disk and upload it as an attachment to the VSTS work item. Finally, I’m replacing the original image source link with the new VSTS image link. Good to know is that once in-line images are detected inside the html field, those images are stored on the server and the temporary image file attachments may be deleted as well.

 


Retain VSTS Build Indefinitely – Fetch Build ID from RM artifact variable

June 26, 2016

While showing a Visual Studio Release Management demo in a Practical DevOps training, I stressed how important it was that the build artifact, which was used during a release, was not destroyed by the built-in retention policy. By default, the output of a build run is only stored for 10 days. So, in case you really want to keep the build and build artifacts, you must take care of this yourself.

For that purpose I created a powershell script which calls the VSTS REST API to accomplish this.

The powershell script is called from a release management task at the beginning of the release process.

KeepBuildArtifact

The powershell script calls the VSTS Build v2 REST API and uses Basic Authentication (passed in the headers) with a Personal Access Token password.

At the moment I worked on this activity (probably still in private preview of VSRM – September 2015) it was not possible yet to fetch the exact Build ID through the build artifact which is linked in the release definition. That’s why I was dropping a simple text file with the Build ID in the build process which was also stored in the build artifact. That file was then used in the release management process to parse the Build ID.

BuildId

Apparently, this workaround is not necessary anymore and you can now immediately fetch the Build Id from the build artifact via a pre-defined release management artifact variable RELEASE_ARTIFACTS_[source-alias]_[variable-name]. Read more about the available RM artifact variables.

Next task on my todo list => create a VSTS extension to provide a dedicated build/release task.