Tuesday, 29 September 2015

Resolve Common Errors - Run Automated Tests with VS Release Management 2013


There is a tool available in Release Management 2013 to execute automated tests, once the deployment completes. image
image
How to setup this is well explained in article here. http://nakedalm.com/execute-tests-release-management-visual-studio-2013/
With the custom component set as below in release template, you should be able to excute tests.image
Below are few issues you might encounter, and steps to fix them.
*******************************************************************************************************
Error 01:
Cannot process argument transformation on parameter
'PlanId'. Cannot convert value "40426" to type "System.Int16". Error: "Value was either too large or too small for an Int16."  Unfortunately this error comes with the component reporting a “succeed”. But log shows the failure. This error is discussed here.
image
This is obvious error if you have more TFS projects and larger IDs for work items above 32767. You would not be able to use the Microsoft provided tool  image
To Resolve:
You need to get the TCM.ps1 from the tool and edit it. Downloading it from RM tools is not possible simply because downloading or any other modification is not available for Microsoft published tools.image
In the deployment agent machine, deployment agent user’s temporary folder (component log contains this path with the error), you can find the file.
C:\RMT\RM\T\RM\XX-MTM-Test\201509241626076897123-31\TcmExec.ps1
In this example users temp path is set to  C:\RMT. Default path is C:\users\username\AppData\Local\Temp\.. as shown below. (More info here) image
Copy this script and change the Int16 parameters highlighted below to Int.image
image
Then create a new tool in Release Management using the powershell script.New MTM RM Tool
Set arguments as follows.
-File ./TcmExec.ps1 -Title "__TestRunTitle__" -PlanId __PlanId__ -SuiteId __SuiteId__ -ConfigId __ConfigId__ -Collection "__TfsCollection__" -TeamProject "__TeamProject__" -TestEnvironment "__TestEnvironment__" -BuildDirectory "__BuildDirectory__"
Change the component in the template to use the new tool created, instead of provided “MTM Automated Tests Manager”.image
*******************************************************************************************************
Error 02:
If MTM (Microsoft test Manager)  not available in the deployment server below error occurs.“ERROR: Unable to locate C:\RMT\RM\T\RM\XX-MTM-Test\IDE\TCM.exe”image
To Resolve:
It is mandatory to have MTM(Microsoft test Manager) installed on the server that is initiating the tests. For this purpose separate machine is setup with deployment agent and it is installed with Visual Studio Test Professional 2013. It is not recommended to install Visual Studio in the target deployment server (DevInt/QA/UAT servers should be similar to Production, without having Visual Studio in them). Better option is to keep a separate machine attached to each environment to trigger tests after deployment.image
image
This is used in the Release template to execute tests after deployment.image
*******************************************************************************************************
Error 03:
With new tool test execution failed with below error.
“TF30063: You are not authorized to access http://yourtfs:8080/tfs/collection.” image
To Resolve:
Add the deployment agent user configured in test initiating machine, to the Test Service Users in the relevant project collection.image
*******************************************************************************************************
Error 04:
After fixing all above issues, tests initiated after the deployment with release template. image
This time reading results fails with “A Visual Studio testing sku must be installed to use this command.”image
To Resolve:
As discussed in this forum thread only working solution was to install “Visual Studio Ultimate (2013)” in the machine which is acting as deployment agent, triggering the tests using the component in the release template.
*******************************************************************************************************
With all above fixed test were executed successfully and results were shown in the tool log.image image

1 comment:

Richie said...

My test gets initiated on the test agent and runs through the actions, but Release Management still says ========== Test: 1 tests ran, 0 succeeded, 1 failed, 0 inconclusive ==========

ERROR: Operation failed with error code 1.

Done.


Please help.