There is a tool available in Release Management 2013 to execute automated tests, once the deployment completes.
How to setup this is well explained in article here. http://nakedalm.com/execute-tests-release-management-visual-studio-2013/
With the custom component set as below in release template, you should be able to excute tests.
Below are few issues you might encounter, and steps to fix them.
*******************************************************************************************************
Error 01:
Cannot process argument transformation on parameter
'PlanId'. Cannot convert value "40426" to type "System.Int16". Error: "Value was either too large or too small for an Int16." Unfortunately this error comes with the component reporting a “succeed”. But log shows the failure. This error is discussed here.
This is obvious error if you have more TFS projects and larger IDs for work items above 32767. You would not be able to use the Microsoft provided tool
To Resolve:
You need to get the TCM.ps1 from the tool and edit it. Downloading it from RM tools is not possible simply because downloading or any other modification is not available for Microsoft published tools.
In the deployment agent machine, deployment agent user’s temporary folder (component log contains this path with the error), you can find the file.
C:\RMT\RM\T\RM\XX-MTM-Test\201509241626076897123-31\TcmExec.ps1
In this example users temp path is set to C:\RMT. Default path is C:\users\username\AppData\Local\Temp\.. as shown below. (More info here)
Copy this script and change the Int16 parameters highlighted below to Int.
Then create a new tool in Release Management using the powershell script.
Set arguments as follows.
-File ./TcmExec.ps1 -Title "__TestRunTitle__" -PlanId __PlanId__ -SuiteId __SuiteId__ -ConfigId __ConfigId__ -Collection "__TfsCollection__" -TeamProject "__TeamProject__" -TestEnvironment "__TestEnvironment__" -BuildDirectory "__BuildDirectory__"
Change the component in the template to use the new tool created, instead of provided “MTM Automated Tests Manager”.
*******************************************************************************************************
Error 02:
If MTM (Microsoft test Manager) not available in the deployment server below error occurs.“ERROR: Unable to locate C:\RMT\RM\T\RM\XX-MTM-Test\IDE\TCM.exe”
To Resolve:
It is mandatory to have MTM(Microsoft test Manager) installed on the server that is initiating the tests. For this purpose separate machine is setup with deployment agent and it is installed with Visual Studio Test Professional 2013. It is not recommended to install Visual Studio in the target deployment server (DevInt/QA/UAT servers should be similar to Production, without having Visual Studio in them). Better option is to keep a separate machine attached to each environment to trigger tests after deployment.
This is used in the Release template to execute tests after deployment.
*******************************************************************************************************
Error 03:
With new tool test execution failed with below error.
“TF30063: You are not authorized to access http://yourtfs:8080/tfs/collection.”
To Resolve:
Add the deployment agent user configured in test initiating machine, to the Test Service Users in the relevant project collection.
*******************************************************************************************************
Error 04:
After fixing all above issues, tests initiated after the deployment with release template.
This time reading results fails with “A Visual Studio testing sku must be installed to use this command.”
To Resolve:
As discussed in this forum thread only working solution was to install “Visual Studio Ultimate (2013)” in the machine which is acting as deployment agent, triggering the tests using the component in the release template.
*******************************************************************************************************
With all above fixed test were executed successfully and results were shown in the tool log.
