TFS Build Agent Fail on Get Source After Upgrade

TFS on premise requires you to keep your TFS updated as almost in every quarter a new update to the TFS is released as well as a new version every year. For this you might have to do TFS Upgrades. If you have upgraded from TFS 2015 to TFS 2018 you have to remove the version 1.xx build agents and configure version 2.xx build agents. Your existing build might encounter workspace issues and may not be able to get sources downloaded to build, in the build agents after the upgrade of agents done. Let’s look at how we can get this issue resolved.

Updating TFS Build Agents from Version 1.xx to 2.xx After TFS Upgrade 2018 from TFS 2015

If you have upgraded the TFS from TFS 2015 to TFS 2018 you will encounter a situation where your vNext build agents 1.xx are no longer valid. You will get a message in the Agent Pool/Queue indicating that the agents are deprecated and you need to migrate them. Let’s understand the steps required to migrate your TFS build agents from 1.xx to 2.xx.

Script to Delete All Workspaces of a Computer from TFS

Sometime you need to clean up machines, especially build servers after a TFS upgrade to get the agents working as intended. If you ever come across a situation where you want to remove all Team Foundation Version Control (TFVC) workspaces from TFS for a given machine, considering all collections and all users, the script available here will help you to do that.

TFS 2015.2 to TFS 2018 Upgrade–Lesson Learnt

It is great if the latest updates to TFS can be applied as and when they are released. But for a large organization it might not be sometimes easy. There may be few version gaps when you try to upgrade your TFS. Let’s discuss a problem with SQL server, faced while upgrading from TFS 2015.2 to TFS 2018, and how can it be fixed without getting into deeper troubles. This upgrade was done after a pre production trial using a clone of the TFS 2015.2, and no issues faced during TFS2015.2 from to TFS 2018 in the trial steps, which were exactly followed in production, except for clone TFS 2015.2 of course. Still in a production scenario you might run into unexpected .

Creating Multiple Artifacts in a Single Build

Sometime you may want to create multiple artifacts in a build for separate each project output into different artifact. This question was asked in a forum “To create CI/CD pipeline to deploy web application and windows service on different VMs there is a need to create separate artifacts, for both project. How artifacts for all projects can be published separately in a CI pipeline ?” To answer that question, let’s look at very simple implementation possibility in VSTS/TFS builds.

Running UI Tests with Unified Agent

You needed to setup dedicated machines for testing when using Run Functional Tests and you had to execute, Deploy Test Agent step and Copy Files step as prerequisites. With the introduction of unified agent enabling build, deployment and testing using same agent, now it is possible to setup pool of machines as Test clients to cater multiple project needs. You can even deploy and test using the same agent if you wish to do so.  Unified agent support is available to run tests as multi-machine distributed execution. You do not require to execute copy files task as artifacts downloaded by the agent can be used for running tests. Let’s look at how to prepare a unified agent for running UI tests.

PS Script Get All Team Projects Listed as HTML–TFS/VSTS

You may want to sometimes satisfy the needs of your team, specially PMs, requesting already available information in document format. Some managers are lazy to go and explore (that's why they choose to be managers not techies) and they sometimes expect information to be delivered to their hands . List of Team projects in each collection can easily be seen on the TFS web interface or in VSTS in admin mode. But here is a PowerShell script to generate the same list as HTML file with link to the relevant team project, if someone requested you to have it as a separate document.

Build Definition as Code with YAML Builds

Build configuration as code and store it in version control was a request from some time back. It is great to have your build definition as code alongside your application source code. Microsoft has now released preview of YAML builds for this purpose to VSTS. Even though as of now it is limited to use with VSTS GIt and GitHub, version control (TFSVC not supported) to create CI builds, it is just the beginning of a new way to think about defining CI, CD pipelines and it will evolve into much greater levels in the future. Let’s get a basic YAML build done to understand what it is.

Moving TFS 2005 Collection to TFS 2013.3 – Plan, Execution & Lessons Learnt

Many organizations still use old platforms and they are reluctant to move to new platform with fear of failure in doing the move. But hanging onto older products would not give any benefit as well as it would eventually fail to meet the demands of the modern business and software development requirements. Visual Studio Team System 2005 (TFS 2005) is such old tool, but still people using it for the production work. Let’s have a look at steps taken to move a TFS 2005  as a collection into TFS 2013.3 (again this is not the latest version, but this client demand was to get it to 2013.3), and the thing to keep an eye on to avoid any issues in the move.

TLS 1.2, PowerShell and Dynamics CRM with VSTS Release Management

PowerShell often used in tasks in VSTS release management. When accessing services authenticated via AAD (Azure Active Directory) it might require to use TLS 1.2 protocol, if it is configured to be required in AAD setup. Setting up the machine as per instructions here to allow TLS 1.2 may not work with PowerShell. In case you are accessing Dynamics CRM online with PowerShell using Microsoft.Xrm.Data.PowerShell in VSTS release management you might see this issue as  “[error]Get-CrmConnection : The pipeline has been stopped.”

Trigger Release from PS in TFS 2015.2

Can you trigger a release in TFS 2015.2 from another release definition execution? Was the request from one of the teams. To fulfill this requirement, if you are using VSTS or TFS2017,  you can use the extension “VSTS Trigger” available in marketplace. But this is not usable in TFS 2015.2 as it does not have the Personal Access Token feature. The PowerShell script described in this post will let you trigger a release from another release.

Mistake in .sln Can Cause NuGet Restore Task in Build to Fail and Hide the Real Issue

NuGet packages are very commonly used in VS Development. Using build VSTS task NuGet Restore is general practice of downloading packages while doing the automated builds. Today suddenly a team ran into the same issue described here https://developercommunity.visualstudio.com/content/problem/97737/error-da-toolnuget400x64nugetexe-failed-with-retur.html. Let’s have a look at the real issue caused this error in this particular teams’ instance and how it was found and resolved.

Setting Quality Profile via VSTS/TFS Build for SonarQube

Sonar analysis helps to identify the technical debt in the source code of your project. Running it integrated with VSTS or TFS builds add more value by automating the quality checks with sonar server. You can define your own custom Quality Profile to analyze your source code with selected set of rules. Let’s look at how you can specify the Quality Profile with the TFS/VSTS build task, that can be used to automate scanning of your code, when using the SonarQube extension in the visual studio marketplace.

Code Review with Git in Team Services

Code Review plays important part in software development as it helps to identify issues that may be overlooked. Reviews on code improve overall quality of the delivered software, and reviews are done in different ways by the teams, at times using pair programming, peer review and formal inspection of code by team leads etc. Team Services Git  version control now facilitates Code Review, via pull requests.

Business Rules for Work Items in Team Services

For inherited process model in VSTS, you can use Custom Rules to define rules that can clear the value of a field, copy a value into a field, and apply values based on dependencies between different fields' etc. You can define actions against conditions using custom rules. This provide you the flexibility in VSTS to go beyond the level of setting a default value or making a field required. To customize work items in an existing team project of Hosted XML model for VSTS, or on-premises TFS, you have to customize, downloaded process template or individual work item XML files (using witadmin tool) respectively. Differences between process models are described here.

TFS DB Restore–DB Set to Suspect

When you have configured backups with TFS admin console and restore them in case of a need, you are expecting, your TFS databases to restore without any issues. But you might run into this situation if you just, missed simple thing to check. For example the target server SQL version, should exactly match to the source or your production TFS, SQL version. Let’s have a look at learning experience I went through due to a slight mistake, which taught me how much vigilant you need to be when you are doing the, administrative work with TFS.

Changing Work Item Icons – TFS vs VSTS

It is now possible to change the work item icons  in on premises TFS  2017.2 and in VSTS. This is a cool feature allowing you to, recognize the type of work item at a glance. However there is a significant difference  between TFS and VSTS how you apply this change. Let’s explore how we can change icon for existing and any new tem projects in both TFS 2017.2 (works just same in TFS 2018 RC1) and in VSTS.

Add Reporting Services to an Existing Team Project in TFS 2018 RC1

From TFS 2015 it was possible to create Team Projects for on premises TFS, using the web interface. But when you create a team project from web interface it does not create Reporting Services or SharePoint integrations. With TFS 2018, SharePoint server integration has been removed and if you want to integrate with SharePoint, setup your SharePoint site using the SharePoint’s own interface. However, Reporting Services is still in use with TFS 2018, and integration to Reporting Services gets created when a Team Project created via Visual Studio (you can use VS 2017 to create a Team Project in TFS 2018 RC1). A Team Project created with web interface will not have a Reporting Services integrated and let’s look at how to create that integration using command line.

Download Offline Installer for VS 2017

There is no ISO available to download for VS2017. You can view why Microsoft did not make ISO available, is discussed in here. However if you want to download VS 2017 to perform an offline installation you can follow the steps described below.

• Download the relevant VS installation .exe (for example vs_enterprise__984310617.1489458965.exe).
• Copy it to your local hard drive folder let’s say E:\Install
• Create a folder to download installation files say E:\Install\VS2017 (can be in a different path such as D:\VS2017)
• Open command prompt as administrator and change the directory to where your downloaded .exe is available
• Execute the command below

TFS 2018 RC1 – Installation in Windows Server 2012

TFS 2018 RC1 is now available and you can download it from https://www.visualstudio.com/downloads/. Release note here explains the new features you can expect in TFS 2018. Below features are going to be removed from TFS with this version.

• SharePoint Extensions
• XAML Builds
• Team Rooms
• Lab Center and automated testing flows in Microsoft Test Manager support

Requirements for TFS 2018 can be found here.  Let’s look at the installation experience with TFS 2018 RC1 on Windows 2012 Server with SQL 2016 SP1.

Cloning Your TFS Server Part 03 – Configuring the AT

To clone an instance of a TFS server while the current live instance is running it is required to take backups as explained in “Cloning Your TFS Server Part 01 – Backing Up and Restoring Databases” and restore the DBs in new hardware. After that there should be preparation activities run as described in “Cloning Your TFS Server Part 02 – Prepare Restored Databases” to make sure the current and cloned TFS instances are not getting corrupted, when both running simultaneously in the same network. Let's go through the configuration steps to configure the cloned TFS instance.

Cloning Your TFS Server Part 02 – Prepare Restored Databases

You have to make sure you are not corrupting your current TFS instance that is up and running, while you are cloning a  TFS instance. For this you need to prepare the restored databases of the TFS, in new hardware, before configuring the cloned Application Tier of TFS. In the “Cloning Your TFS Server Part 01 – Backing Up and Restoring Databases” how to back up and restore TFS database was discussed in detail. Let’s look at each of the preparation steps required in order to guarantee both current TS instance and cloned instance can coexist in the same network.

Cloning Your TFS Server Part 01 – Backing Up and Restoring Databases

You might want to clone your TFS instance if you are doing a hardware move upgrade. There can be other situations such as you need to move one of your projects to VSTS (Team Services) using high fidelity database import, without migrating other projects in the same collection. If you prefer to keep your current TFS untouched while trying to create an isolated collection for a single project, you might want to clone and create an instance of your TFS. There can be several other experimental reasons you may have to clone a TFS instance.

TFVC History Report with PowerShell

Team Foundation Version Control (TFVC) is widely used with Team Foundation Server and in Team Services, as a source control system to manage source code repositories.Usage of TFVC can be found here. If you want to generate history report for TFVC including details of files changed and any work items associated with the changesets, you can use the script available at TechNet Gallery.

Show Hidden Fields with Empty Values for Edit - Kanban Cards - Team Services

With the latest release to Team Services Kanban Card has been added with a new feature. Previously, if you want to edit a value in a field that is currently empty, in the Kanban Card You  had to keep it showing even if value is empty. With the new feature available now you can let Kanban Card to hide a field, if it is empty and expand the Card to show empty fields  so that you can edit them.

New Release Definition Editor–Team Services

Release definition in Team Services helps you to define deployment pipelines across target environments. You can setup a release definition to deploy via agents, deployment groups or execute agent less activities. With the latest update to Team Services a new release definition editor is introduced. Let’s ex[lore the new editor.

Sizing Backlog Items with Team Services

Backlog item sizing will help an Agile team to identify their velocity and plan for future iterations/sprints considering it. It is important the team members have discussed a backlog item and have a common understanding on what needs to be done before it is relatively estimated. Sizing for backlog items in Team Services work items such as User Story, Bug can be updated in Story Points or Effort field depending on the template you are using. Team Services extension, “Estimate” is available in Visual Studio Marketplace allow you to do effective estimating of backlog item, by enabling team to vote for each work item using on Team Services, while getting Story Points or Effort field update automatically with the size value, once team is committed to a size.

Ignoring Visual Studio 2017 Created Files in Git

Visual Studio 2017 creates few files such as slnx.sqlite, .suo and these files often give trouble if you get it added to a branch in Git version control. Every time you open a solution in Visual Studio 2017, these files are indicated as pending changes and you are unable to do a pull, push etc. without undoing or committing these files. This is annoying since you actually do not want these to be added as changes to version control repository. You can try completely removing the files not needed from repository but if you are not willing to go through somewhat risky procedure, what options you have for solving this problem?

Let’s look at how to fix this issue.

.gitignore

Adding unwanted files to .gitignore works as long as you have not initially committed them to any branch. But in this case where it is already committed adding .gitignore entry such as /.vs/slnx.sqlite does not help.

Ignore changes to the files

You can run below command to ignore the changes to these files by assuming they are unchanged.

git update-index --assume-unchanged filename

To run the command open command prompt from team explorer window of Visual Studio by right clicking on the repository name of the connect page.

The command git update-index --assume-unchanged slnx.sqlite gives error “fatal: Unable to mark file slnx.sqlite” because it actually cannot find the file.

To find the correct paths you can execute git ls-files command and use the path listed to mark the file. You can list git untracked files as well by executing git ls-files -o but untracked files cannot be marked. Adding them to gitignore would be sufficient.

Providing correct path to file from the location you are running the command will make it execute without error message.

slnx.sqlite file no longer considered as changed.

You can undo the the assume-uchanged state by executing command below.

git update-index –-no-assume-unchanged filename

Team Services - Custom In Progress Sate to Completed State – Closed Date Not Updating Issue

Update! – Issue described here is now resolved in Team Services and it has even included fix for last column order issue as well.

Team Services now allows work item states to be customized and you can change the workflow to your preference. This is explained in the previous post “Custom States for Work Items in Team Services with New Process Customization Experience” in detail. As explained there you can hide available default states and use your custom sates instead of them. But as of now state change from a custom state to Completed category state does not update the Closed Date and Closed By fields in Bug and User Story/PBI work items. For Task work item in custom state , changing to Completed category state, updates Closed Date but not the Closed By field.

Let’s further understand this issue with an example.

Custom States for Work Items in Team Services with New Process Customization Experience

Custom states for work items could be introduced and usage of new custom workflow for state transitions, is possible in team services from some time back. The post here describes how to customize work item states. With latest updates to Team Services a new process customization experience is introduced as described in “Team Services Process Customization”. Let’s  explore the state workflow customization of work items with the new process customization experience.

Team Services Process Customization

Team Services is coming up with many improvements and one of it is the new process customization experience introduced recently. This allows you to have your own custom work item types, custom work item state flows and custom fields for work items etc. It is made easier to start customizing even directly from work item form. Let’s look at and understand process customization options available for you to optimize your work process.

Deploying Web Apps to Azure Virtual Directories

My friend Jaliya Udagedara has written nice post here, explaining how to deploy a Core web application into Azure App Service Application virtual directory. Below is comparison of few more settings you can use with Azure virtual directories.

Deployment Groups in VS Team Services

Deployment groups provide robust out-of-the-box multi-machine deployment with team services release management. It allows you to run deployments across multiple machines. You can install agent on each of your target servers directly and run rolling deployment to those servers, unlike agent based deployments where you install build/deployment agents on proxy servers in an agent pool. You can use all tasks in task catalog on your target machines.

Deployment group allows you to create target deployment machine groups, without requiring you to register machines in agent pools or queues. Machine in a deployment group will have an agent registered with the named deployment group, within the team project providing required isolation for target environment (such as DevInt, QA, UAT, Demo, Production etc.), for a given product or project.

Logical group of targets (machines) having an agent installed on each of them is a deployment group. It represents your physical environments. It can be single-box, multi-machine or a farm of machines. You can specify the security context for your physical environments by using the deployment groups.

Let’s look at how to create a Deployment Group in Team Services.

string.Format & { }

C# developers often use string.Format in handling string. string.Format the { and } is used as a special character to identify the parameters.

For example:

scrollDepth = “10”

string.Format("DOWN {0}",scrollDepth)  generates a string DOWN 10.

But what if a { should be used as a value in the string. Is there a way to include { in the string?

To escape a { you can {{. So if you want to get the resultant string as “{DOWN 10}” the syntax to use is

string.Format("{{DOWN {0}}}",scrollDepth)

Auto Generating CD Build/Release Pipeline Targeting Azure App Service for MVC Web Application

VSTS allows great flexibility with it’s new web based build and release management features. You have several templates which can be used to create build and release definitions you need quickly. Lots of out of the box tasks allow you to further customize your definitions to support the steps that you need to perform during a build or a deployment. Visual Studio Marketplace is getting more and more components added to it, allowing you to find a task for almost any build deploy step you may require to perform. Taking the steps to the next level, you can now even generate your build/release pipeline with Visual Studio 2017, after adding the extension Continuous Delivery Tools for Visual Studio.

Let’s walkthrough the steps of generating a CD pipeline for an MVC Web Application, to allow it to be deployed to Azure App Service.

Enable Conditional Build/Release Tasks in VSTS

TFS/VSTS new build and release management is providing great flexibility in implementing CI,CD pipelines. To further enhance the flexibility a new feature “Conditional Tasks” is introduced to Tasks of build and release management. This was a user voice request that got implemented and available as preview in Team Services. This new conditional control option let’s you execute a task base on few predefined conditions and using custom conditions.

Predefined conditions

• Only when all previous tasks have succeeded
• Even if a previous task has failed, unless the build was canceled
• Even if a previous task has failed, even if the build was canceled
• Only when a previous task has failed

 

Building ASP.NET Core 1.1 Web App with Team Services

ASP.NET Core 1.1 is available with Visual Studio 2017 and it is based on .csproj to provide us with a consistent experience inside Visual Studio. .csproj will be the way forward with ASP.NET Core and Microsoft encourages everyone developing on ASP.NET Core to move to Visual Studio 2017. You can see the recommendation in https://blogs.msdn.microsoft.com/dotnet/2017/03/07/announcing-net-core-tools-1-0/ quoted below. Microsoft will not be updating VS 2015 based tools for .NET core anymore.

Setup yo team

yo team is a great NPM package by Donovan Brown, to automate creation of build and release pipelines. You can use it to generate app with CI/CD in Team Foundation Server 2017 or Visual Studio Team Services. In this post let’s look at how to get it installed, for you to use it to generate CI/CD pipelines

The first requirement is to install node.js and npm latest. You can download it from https://nodejs.org/en/download/. Instruction on installing on windows is here. You can verify the installed version of node and npm by running below commands.

Resolve - TFS Build Hang When Building Wix Projects

Wix tools can be used to generate msi installable packages, .exe setup bundles, .msm merge modules, and .msp patches. Getting a wix project built with visual studio is fairly simple. If you are using VS you can install the extension from marketplace and install wix tools to get going in the development environment. Tools available for VS 217, 2015, 2013, 2012 and 2010.

You can setup TFS builds to build wix projects following the instructions in http://wixtoolset.org/documentation/manual/v3/msbuild/wix_with_team_build.html. There is more detailed documentation available in http://wixtoolset.org/documentation/manual/v3/msbuild/. The blog here also useful http://programcsharp.com/blog/post/building-wix-with-msbuild.

However, when you are running a build to package .msi etc using wix project you might notice the build sometimes hangs forever. This happens because of a deadlock when msbuild and wix waiting on the output stream.

Sonar Server on Azure VM

SonraQube has become grate tool providing detail analysis of code quality. Tracing of history of code quality checks let the development teams identify how the code quality improved (could be how the code is gotten so bad even). How to setup a SonarQube server to use with team foundation server is described in the post “Setting Up Sonar Server for TFS”. Let’s look at steps required to set this up in an Azure virtual machine and expose it to use with team services account.

1. Setup an Azure virtual machine with Windows Server 2012 R2 Datacenter.
2. Follow the instructions in the post  “Setting Up Sonar Server for TFS” and get the SonarQube server running inside the virtual machine.
3. To allow accessing the Sonar server default port 9000 through firewall of the virtual machine add an inbound firewall rule in the virtual machine allowing port 9000 for public and private access.
4. Above step alone will not allow public access to Azure VM hosted sonar server. You need to allow accessibility in Azure nsg(network security group) for the port. To do this go to the virtual machine in Azure portal, select Network Interfaces and click on the network interface.

Resolve MSB4019: The imported project "C:\Program Files\dotnet\sdk\1.0.0-rc4-004771\Microsoft\VisualStudio\v14.0\DotNet\Microsoft.DotNet.Props" was not found or Unable to start the process. No executable found matching command "dotnet-projectmodel-server"

You might encounter compilation error in Visual Studio 2015 when a new .NET Core 1 web app is created, if you have installed Visual Studio 2017 (RC as of now). When a new .NET Core 1 web application is created/opened in Visual Studio 2015, with the project template you get with .NET Core 1.0.1 tools Preview 2, while having ".NET Core cross-platform development" for Visual Studio 2017 in your machine, you might see below error prompted.
---------------------------
Microsoft Visual Studio
---------------------------
The following error occurred attempting to run the project model server process (1.0.0-rc4-004771).
Unable to start the process. No executable found matching command "dotnet-projectmodel-server"
The project model server process provides intellisense, build, and reference information to Visual Studio and without it your experience will be very limited. Please try closing and reopening Visual Studio to see if that corrects the problem. Alternatively, check that the .NET Core SDK is properly installed.
---------------------------
OK  
---------------------------
When attempt to compile the solution you may encounter a compilation error below.
WebApp.xproj(7,3): error MSB4019: The imported project "C:\Program Files\dotnet\sdk\1.0.0-rc4-004771\Microsoft\VisualStudio\v14.0\DotNet\Microsoft.DotNet.Props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

SonarQube Extension for VSTS/TFS

In the post Setup SonarQube Analysis with TFS Builds usage of Sonar Server with TFS build to measure quality of code is explained. The task catalogue tasks explained in the post Setup SonarQube Analysis with TFS Builds are now deprecated. There is an extension available in marketplace for SonarQube. This extension provide new begin analysis and end analysis tasks to be used in a TFS build.
Once you install the SonarQube extension for VSTS/TFS you can setup service endpoint type of SonarQube.

Bulk Editing Work Items in TFS

There was a question from a team member how to bulk edit TFS work items. In the past the only answer to this question would be use excel. Open a query in Excel and update and publish. Is there any other way to achieve this easily without doing one by one work item edit in a query result? Query result opened in Web or in Visual Studio you can edit each work item by clicking on each, and then save bulk of work items. But is it possible to change a value of a field for many work items at once. Let’s look at the bulk edit options available with TFS see the possibilities.

Trigger a Release Based on Build Tag– VS Team Services

Visual Studio Team Services now supports triggering a release, in team services release management, when a defined tag is available in a completed build. This helps you to filter a build or builds, to get deployed via the release pipeline, out of builds that are successfully completed.
For example a build that is tagged as RTM, would trigger a release while another build, completed successfully, but with no tag is not triggering a deployment. In order to say a release should only be triggered with a given tag, it should be defined in the Triggers tab of the release definition.

Resolving “Deployment on this environment was cancelled.”

You might encounter “Deployment on this environment was cancelled.” when you have setup a new build/release agent and try to do a deployment, with on premise TFS. It is a confusing error message and not enough information at all to identify the issue. How to identify where the problem is easier if the agent is on premise. Then you have the ability to look at additional diagnostic logs. Lets look at one example scenario you are getting this error.