Monday, 10 October 2016

Setup Test Farm and Test Clients for Test Execution

Let’s look at steps required for Setup Test Farm, Start and Stop, and Test Clients for Test Execution for setting up a Test Farm as Azure VMs–For TFS2015 Release Management(This is the 05th step) and other four steps are listed below.
  1. Create Virtual Network in Azure
  2. Create Virtual Machines in Azure for the Test Farm.
  3. Setup release agents in the admin machine in Azure, to use with each test client following instructions in here.
  4. Enable PowerShell remoting in each test client and verify access from the test farm admin VM in Azure.
For setting up VM start and stop you need to set resource manger service link to azure with TFS following the instructions here.
Setup to start and stop Resource Groupimage
image
Use a blob in azure to keep the status of the test client. Once test client done create a text file as done to notify the farm stop environment. This is required since TFS 2015 is not yet able to wait for multiple environments in release management to start its execution. (have a look at Release Management - Deploy based on conditions in multiple environments in here).image
Script to do this

param(
    [Parameter(mandatory=$true)]
    [string]$storageAccountName,
    [Parameter(mandatory=$true)]
    [string]$ResourceGroupName,
    [Parameter(mandatory=$true)]
    [string]$releaseContainerName
    
)

#Add-AzureRmAccount 

$releaseContainerName = $releaseContainerName.ToLower();

$StorageAccount = Get-AzureRmStorageAccount  -StorageAccountName $storageAccountName -ResourceGroupName $ResourceGroupName  #-StorageAccountKey $storageKey

Write-Host "Storage Account obtained"
$StorageAccount

$NewReleaseContainer = New-AzureStorageContainer -Name $releaseContainerName -Context $StorageAccount.Context -Permission Blob

Write-Host ("Release container:{0} created in storage account:{1}" -f  $releaseContainerName, $storageAccountName)

$NewReleaseContainer
Then use a wait script to see if the test environments done in stop environment.image
Wait script

param(
    [Parameter(mandatory=$true)]
    [string]$storageAccountName,
    [Parameter(mandatory=$true)]
    [string]$ResourceGroupName,
    [Parameter(mandatory=$true)]
    [string]$releaseContainerName,
    [Parameter(mandatory=$true)]
    [string]$releaseEnvName
)

#Add-AzureRmAccount 
$releaseContainerName = $releaseContainerName.ToLower();

$StorageAccount = Get-AzureRmStorageAccount  -StorageAccountName $storageAccountName -ResourceGroupName $ResourceGroupName  #-StorageAccountKey $storageKey

Write-Host "Storage Account obtained"
$StorageAccount

$releaseContainer = Get-AzureStorageContainer -Name $releaseContainerName -Context $StorageAccount.Context 

Write-Host ("Release container:{0} obtaned in storage account:{1}" -f  $releaseContainerName, $storageAccountName)

$BlobContainer =  $releaseContainer.CloudBlobContainer;

Write-Host "Blob Container obtained"

$BlobContainer


$releaseEnvBlob =  $BlobContainer.GetBlockBlobReference($releaseEnvName)
Write-Host ("Release env:{0} blob reference obtained." -f $releaseEnvName)

$releaseEnvBlob.UploadText("Done");
Write-Host ("Release env:{0} done flag set." -f $releaseEnvName)
Remove blob container script

param(
    [Parameter(mandatory=$true)]
    [string]$storageAccountName,
    [Parameter(mandatory=$true)]
    [string]$ResourceGroupName,
    [Parameter(mandatory=$true)]
    [string]$releaseContainerName
    
)

#Add-AzureRmAccount 
$releaseContainerName = $releaseContainerName.ToLower();

$StorageAccount = Get-AzureRmStorageAccount  -StorageAccountName $storageAccountName -ResourceGroupName $ResourceGroupName  #-StorageAccountKey $storageKey

Write-Host "Storage Account obtained"
$StorageAccount

Remove-AzureStorageContainer -Name $releaseContainerName -Context $StorageAccount.Context -Force

Write-Host ("Release container:{0} removed in storage account:{1}" -f  $releaseContainerName, $storageAccountName)
Log on to to azure VM setting up as agent and create shadow user. add to admin.image
image
http://chamindac.blogspot.com/2016/06/deploying-to-untrusted-domaintfs-2015.html
Machines created with same virtual network, enable file and printer sharing.. since same workgroup all will be accessible Same username password as administrator user.(Even different user would work.). All of the machines are in same workgroup in Azure (refer Create Virtual Machines in Azure for the Test Farm).image
Setup the test agent deployment task and test execution task.image
Windows file copy task to copy the test binaries to the test execution folder.image
Test Run task to run the tests.image
Once a test client is done it will be creating a text file in blob container to notify the waiting task monitoring the blob container.
Set environment done script.

param(
    [Parameter(mandatory=$true)]
    [string]$storageAccountName,
    [Parameter(mandatory=$true)]
    [string]$ResourceGroupName,
    [Parameter(mandatory=$true)]
    [string]$releaseContainerName,
    [Parameter(mandatory=$true)]
    [string]$releaseEnvName
)

#Add-AzureRmAccount 
$releaseContainerName = $releaseContainerName.ToLower();

$StorageAccount = Get-AzureRmStorageAccount  -StorageAccountName $storageAccountName -ResourceGroupName $ResourceGroupName  #-StorageAccountKey $storageKey

Write-Host "Storage Account obtained"
$StorageAccount

$releaseContainer = Get-AzureStorageContainer -Name $releaseContainerName -Context $StorageAccount.Context 

Write-Host ("Release container:{0} obtaned in storage account:{1}" -f  $releaseContainerName, $storageAccountName)

$BlobContainer =  $releaseContainer.CloudBlobContainer;

Write-Host "Blob Container obtained"

$BlobContainer


$releaseEnvBlob =  $BlobContainer.GetBlockBlobReference($releaseEnvName)
Write-Host ("Release env:{0} blob reference obtained." -f $releaseEnvName)

$releaseEnvBlob.UploadText("Done");
Write-Host ("Release env:{0} done flag set." -f $releaseEnvName)
When release run you might run into below issue.
##[warning]DistributedTests: Task 'DownloadTestAgent' for machine do-tf-tc01:5985's Error : 
System.Management.Automation.Remoting.PSRemotingTransportException: 
Connecting to remote server do-tf-tc01 failed with the following error message : 
The WinRM client cannot process the request. If the authentication scheme is different 
from Kerberos, or if the client computer is not joined to a domain, then HTTPS transport 
must be used or the destination machine must be added to the TrustedHosts configuration 
setting. Use winrm.cmd to configure TrustedHosts. Note that computers in the TrustedHosts 
list might not be authenticated. You can get more information about that by running the 
following command: winrm help config. For more information, see the 
about_Remote_Troubleshooting Help topic.
image
This is because PowerShell remoting is not setup for test agent machine. Follow instructions in Enable PowerShell remoting in each test client and verify access from the test farm admin VM in Azure. to setup PowerShell remoting. Create directory in the test client from admin machine to verify.image
image
Still you may run into below issue if you have more than one test client machine.
Unable to resolve path $env:SystemDrive\TestAgent. Connecting to remote server do-tf-tc01 
failed with the following error message : The WinRM client cannot process the request. If 
the authentication scheme is different from Kerberos, or if the client computer is not joined
to a domain, then HTTPS transport must be used or the destination machine must be added to the 
TrustedHosts configuration setting. Use winrm.cmd to configure TrustedHosts. Note that 
computers in the TrustedHosts list might not be authenticated. You can get more information 
about that by running the following command: winrm help config. For more information, see the 
about_Remote_Troubleshooting Help topic. ---> System.AggregateException: One or more errors 
occurred. ---> System.Management.Automation.Remoting.PSRemotingTransportException: 
Connecting to remote server do-tf-tc01 failed with the following error message : The WinRM 
client cannot process the request. If the authentication scheme is different from Kerberos, 
or if the client computer is not joined to a domain, then HTTPS transport must be used or the 
destination machine must be added to the TrustedHosts configuration setting. Use winrm.cmd to 
configure TrustedHosts. Note that computers in the TrustedHosts list might not be 
authenticated. You can get more information about that by running the following command: 
winrm help config. For more information, see the about_Remote_Troubleshooting Help topic.
image
to verify from the admin machine execute below command
Get-Item WSMan:\localhost\Client\TrustedHosts
Only last machine added to trusted hosts available.image
To fix Add all machines (http://stackoverflow.com/questions/21548566/how-to-add-more-than-one-machine-to-the-trusted-hosts-list-using-winrm)
Set-Item WSMan:\localhost\Client\TrustedHosts -Value "do-tf-tc01,do-tf-tc02,do-tf-tc03"
image
Test agent deployment cannot copy files error
System.AggregateException: Failed to execute the powershell script. Consult the logs below for details of the error.
2016-09-09T06:25:55.6714118Z ##[warning]Failed to connect to the path \\do-tf-tc03 with the user do-tf-tc03\doadmin for copying.System error 53 has occurred.
2016-09-09T06:25:55.6714118Z ##[warning] The network path was not found.
2016-09-09T06:25:55.6714118Z ##[warning]
2016-09-09T06:25:55.6714118Z ##[warning] ---> System.Management.Automation.RuntimeException: Failed to connect to the path \\do-tf-tc03 with the user do-tf-tc03\doadmin for copying.System error 53 has occurred.
image
Fix by allowing file and printer sharing in test client machine firewall.image
Once test agent machine restart this is causing issues. image
[Window Title]
DTAExecutionHost.exe
[Main Instruction]
DTAExecutionHost.exe has stopped working
[Content]
A problem caused the program to stop working correctly. Windows will close the program and notify you if a solution is available.
[Close program]
Disable to resolve – Not a good fix at all.
image
Another problem asking for login while trying to execute tests
image
image
To resolve
Setup a local account in Test Client and in TFS AT say tfstestsvcshadow. (This is used as the service account for Test Agent.image
image
image
Add this user to team project collection test service account.image
This resolves DTAExecution startup error as well. Enable the disabled DTAExecution in startup.
With this you will be able to execute the tests.image
[image%255B41%255D.png]
Look at Test Farm as Azure VMs–For TFS2015 Release Management to see how this is running.