Duration of a Full Backup Job in TFS

April 24, 2017

In order to properly plan a production migration of TFS I often also check the duration of a full backup. The easiest way to see the evolution of the latest Full Backups is to run a simple SQL Query in the TFS_Configuration database …

TFSFullBackupDuration

SELECT [QueueTime], [StartTime], [EndTime], [Result], [ResultMessage], CONVERT(TIME,[EndTime] – [StartTime]) as Duration  FROM [Tfs_Configuration].[dbo].[tbl_JobHistory] jh JOIN [Tfs_Configuration].[dbo].[tbl_JobDefinition] jd on jh.JobId = jd.JobId WHERE jd.JobName = ‘Full Backup Job’ order by HistoryId desc

Advertisements

Upgrade TFS Team Project features

April 20, 2017

When upgrading TFS, the existing Team Projects won’t automatically adopt the new features of the new TFS version. Some of the new features might require some updates to the Team Project. Note that this will only be required for TFS Team Projects … VSTS Team Projects are automatically updated with each service upgrade.

You can perform this update yourself via the Configure Features wizard. If the Configure Features link is visible for your Team Project, it means that the Team Project requires an update. Otherwise, the new features are already enabled.

alm_cfw_configfeatures

This might work if you don’t have a lot of Team Project Collections and Team Projects. During a recent upgrade to TFS 2017 Update 1 at a customer, I was confronted with 31 Team Project Collections and in total a bit more than 400 Team Projects. No way I was going to hit the configure features link 400 times …

I remember having done this already programmatically in the past (https://www.visualstudio.com/en-us/docs/work/customize/configure-features-after-upgrade#program-updates), but the issue now was that there wasn’t a ready-to-use solution for TFS 2017 Update 1. So, I used some tips & tricks from https://www.visualstudio.com/en-us/docs/work/customize/configure-features-after-upgrade#program-updates and also the Features4tfs CodePlex solution was a good starting point. I wanted to have a scenario where it’s possible to scan a complete TFS 2017 environment with all on-line Team Project Collections and all available Team Projects.

As a result, you can find my solution in Github: https://github.com/pietergheysens/TFSUpgradeTeamProjectFeatures. Because it worked for me with TFS 2017 (Update 1), it doesn’t mean it will work for you. Please test it first during a trial-upgrade and see if it helps you to upgrade your Team Projects in one go.


TFS Production upgrades: stay calm and stick to the plan!

March 4, 2017

When planning a big migration upgrade to TFS 2017 (from TFS 2013) on new hardware, the exact planning of all actions can be very important to make sure the downtime of the TFS environment can be as short as possible and there’s at least some buffer to fix unexpected issues. That’s why I always try to perform production migrations in a week-end and that’s why you should always run a trial migration to have an idea about the total duration.

For this specific migration I’m doing this week-end, TFS 2017 is only an intermediate step because the customer also wants to migrate to VSTS from the TFS 2017 environment. I managed to do this without any issues during a trial run.

So, the plan for the production migration was: bringing the TFS environment offline on Friday evening and already launching the TFS 2017 upgrade wizard on Friday evening to make sure the long upgrade process can continue to run during the night. During the trial upgrade, this process took about 4 hours.

Unfortunately when logging back in on Saturday morning, I noticed the upgrade process failed after more than 3 hours (step 1523 of 1621) due to error TF30042: The log file for the database is full.

UpgradeError

The dedicated log disk on the server was indeed full. Seeing this error might freak you out because first you will believe that the complete upgrade failed and you need to start all over again. This might jeopardize the full plan to have a working VSTS environment on Monday morning.

This is for sure a moment to stay calm and to properly assess the situation and read all text which is available for you in the log file and also have a good look at the warning message in the TFS Upgrade wizard:

One or more project collections failed to upgrade … Start the Administration Console and navigate to the Team Project Collections node to attempt retrying the upgrade for each failed collection.

No need to start all over again! Fix the error which can be found in the error log and try to resume the upgrade process. In my situation I had to clean up the dedicated log file disk before rerunning the job from the TFS Administration Console.

ResumeUpgradeProcess

And indeed, the upgrade process resumed from step 1523 …

I only lost a bit of processing time, but still ok to finish the complete upgrade process before Monday morning …

Having done about 50 TFS upgrades in the last couple of years, I never had to cancel a production upgrade. I always delivered the new environment on time. Of course, there were times were unexpected issues came up or where I needed to perform some aftercare when the new environment was already up-and-running.

Rule #1: always have a backup plan in case of a hard failure

Rule #2: stay calm and properly assess the situation

Rule #3: call help before doing crazy stuff in a production environment


Reporting Services issues after migration to TFS 2015 RC2 (from TFS 2010)

July 10, 2015

Yesterday, I blogged about the migration activities to get to TFS 2015 RC2, but I was still stuck at an error during new Team Project creation.

TeamProjectFailure

Looking into the logs it looked like there was some kind of issue during the upload of the first Reporting Services report “Bug Status” (part of the latest TFS 2015 Agile process template).

TeamProjectFailure-ReportUpload

First I thought there was something wrong with permissions on the Reporting Services site because I also applied a certificate to offer accessibility via https, but that wasn’t it. All seemed to be setup correctly. A sign to look further into the details of the Bug Status .rdl report definition and why it couldn’t be correctly uploaded to the Reporting Services site. After downloading the process template from the Team Project Collection and opening the report with notepad, I knew I was getting closer to the root cause.

DataSourceIssue

As shown in the picture above, the (default TFS 2015) Data Source mentioned in the report definition did not exist in the Report Server database after the migration from TFS 2010 SP1. In the TFS 2010 timeframe, all report defintions were linked to a “2010” Data Source.

Instead of renaming the existing Data Sources (that would break the existing reports, created from a TFS 2010 Team Project), I duplicated the entries to provide extra Data Sources with the expected names for usage with TFS 2015.

Issues resolved. Back on track!


Upgrading TFS 2010 SP1 to TFS 2015 RC2

July 9, 2015

Today I got the opportunity to upgrade a customer from TFS 2010 SP1 to TFS 2015. As blogged by Brian Harry a few days ago, the RTM release of TFS 2015 (not Visual Studio 2015!) is delayed (for a good reason!) and in order to further test the overall migration/upgrade process a new RC2 release has been made available (with go-live license). That’s the build I used today to upgrade from TFS 2010 SP1. The official installation guide is not yet available, so here are some tips to get you started in the right direction … At the customer I got 2 new servers at my disposal with a clean Windows Server 2012 R2 OS. The first server is used for the TFS Application and TFS Data Tier (Single Server Topology). The second one will be used as the TFS Build Server.

Prerequisites

Installing SQL Server 2014 Standard Edition

Before starting the installation of SQL Server 2014, I enabled the .NET 3.5 feature on Windows Server 2012 R2. OSFeature

Required SQL Server features:

SQL2014-FeatureSelection

I always recommend to use dedicated domain service accounts for running SQL Server. In this case I decided to reuse the <TFSSERVICE> account. Note that all services are set to start automatically. ServiceAccountsForSQL

SQL2014-InstallationResult

After successful installation of SQL Server 2014 I started the backup procedure on the old TFS 2010 environment and I copied all SQL .bak files to the new server for restoring all TFS related databases to the SQL Server 2014 instance. SQLDatabases

Configuration of SQL 2014 Reporting Services

Start-RS

Link to the existing restored “ReportServer” SQL Server database.

SetRSDatabase

RS-ExistingDB

RS-SelectReportServerDatabase

RS-Summary

Restore the SQL Server Reporting Services Encryption key.

RS-RestoreEncryptionKey

RS-RestoreEncryptionKey2

Run/Apply the web service and the report manager service.

RS-Webservice

RS-ReportManager

Remove the “old” server from the scale-out deployment configuration [https://intovsts.net/2014/07/14/tfs-migration-upgrade-scale-out-reporting-services-issues/] Test the Report Server website …

RS-Running

Installation TFS 2015 RC2

InstallTFS2015

trialLicense

The trial license is valid for 90 days and can be extended for 30 days. At this moment, there are no product keys available for the RC2 release.

Wizard-Upgrade

Wizard-Upgrade2

Wizard-Upgrade3

Wizard-Upgrade4

No Build Configuration yet.

Wizard-Upgrade5

Wizard-Upgrade6

Wizard-Upgrade7

Wizard-Upgrade8

Wizard-Upgrade9

Wizard-Upgrade10

No SharePoint link (yet).

Wizard-Upgrade11

Verification …

Wizard-Upgrade-Verification

Successful upgrade!

Upgrade-Success

Once the TFS 2015 RTM release becomes available, it’s only a minor upgrade to move to the official RTM bits. Interested in the release notes for TFS 2015? Check out the official news update, posted in April 2015.


Update TFS Build Controller via Powershell

October 21, 2014

During a trial-migration of TFS, I typically prepare a lot of command-line tools to run some background update processes. Instead of writing little C# command-line applications in Visual Studio, I’m moving more and more away from Visual Studio and do stuff via PowerShell.

For a migration upgrade from TFS 2010 to TFS 2013, I have to deal with an update of the Build Controller to a new build machine/server. For a Team Project Collection of 100+ Team Projects and lots of build definitions, this is not something you want to do manually or delegate to all involved dev teams.

Instead of having to start from zero, I found this interesting post from Daniel Mann which does exactly what I want: updating the Build Controller from PowerShell via the TFS API. Except, it does it specifically for one dedicated Team Project.

So, the PowerShell script only needed a bit of tweaking to fetch all available Team Projects via the ListAllProjects method of the ICommonStructureService interface.

param(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]
$TfsUrl,
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]
$NewBuildController,
[Switch]
$WhatIf
)

Function SetBuildControllerForTeamProject($TeamProject, $BuildController, $WhatIf)
{
$buildDefinitions = $buildClient.QueryBuildDefinitions($TeamProject)
$buildDefinitions | % {
Write-Host " >> Checking Build Definition" $_.Name
Write-Host " >>" $_.Name "is using" $_.BuildController.Name

if ($_.BuildController.Uri -ne $controller.Uri) {
Write-Host " >>> Setting" $_.Name "to use $BuildController"
if (!$WhatIf) {
$_.BuildController = $controller
$_.Save()
}
}
else {
Write-Host " >> Build controller is already set. Taking no action."
}
}
}

add-type -Path 'C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\ReferenceAssemblies\v2.0\Microsoft.TeamFoundation.Common.dll'
add-type -Path 'C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\ReferenceAssemblies\v2.0\Microsoft.TeamFoundation.Client.dll'
add-type -Path 'C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\ReferenceAssemblies\v2.0\Microsoft.TeamFoundation.Build.Client.dll'

$tfsUri = new-object System.Uri $TfsUrl
$tpc = new-object Microsoft.TeamFoundation.Client.TfsTeamProjectCollection $tfsUri

$buildClient = $tpc.GetService('Microsoft.TeamFoundation.Build.Client.IBuildServer')
$commonStructure = $tpc.GetService('Microsoft.TeamFoundation.Server.ICommonStructureService')
$controller = $buildClient.GetBuildController($NewBuildController)

$allTeamProjectInfo = $commonStructure.ListAllProjects()
$sortedTeamProjectInfo = $allTeamProjectInfo | sort-object { $_.Name }

foreach($teamProjectInfo in $sortedTeamProjectInfo)
{
Write-Host '************** Scanning Build Definitions in Team Project' $teamProjectInfo.Name
SetBuildControllerForTeamProject $teamProjectInfo.Name $NewBuildController $WhatIf
Write-Host ''
}

You may also use the Community TFS Build Manager (Visual Studio 2013 extension) to modify a number of Build Definitions in bulk, but for my migration scenario, I preferred to have a PowerShell script. Having this script also allows me to modify other settings in the Build Definitions … for example the Build Drop Location.

Community TFS Build Manager


TFS 2010: Virus Scan file/folder exclusions

December 20, 2011

At several customers I have seen that some Virus Scan Systems (no names) have a serious negative impact on the performance of Team Foundation Server. Especially the Build Server may be an easy target for the Virus Scanning actions.

Here’s my latest recommendation I have sent to the Ops Team. Note that all involved servers run Windows Server 2008 R2 (64 bit) and the software is installed on the C drive. The TFS Application Tier also has WSS 3.0 (SP2) installed. Use at your own risk and good luck to get this approved by your Ops Team. Traditionally they handle the presumption of innocence principle: the virus scan is innocent until proven guilty!

  • TFS Application Tier
    • C:\Program Files\Microsoft Team Foundation Server 2010\Application Tier\Web Services\_tfs_data [TFS Version Control Cache folder]
    • C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions [SharePoint Portal]
    • C:\Windows\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files [SharePoint Portal + Web Access]
    • C:\Users\TFSWSSAccount\AppData\Local\Temp [SharePoint Portal]
  • TFS Build Server(s)
    • Build Agent(s) working directory: D:\Builds
    • TFS local cache folder for TFSBUILD account: C:\Users\TFSBUILDAccount\AppData\Local\Microsoft\Team Foundation
  • TFS Data Tier
    • SQL Server data files: .mdf + .ldf
  • Client DevSeat
    • TFS workspaces folder: D:\TFS

I did find some official references: