Chaminda's DevOps Journey with MSFT: Update Script Timeout in MTM Test Settings

You could run into default script timeout of five minutes, while running scripts before running tests using MTM.

Error 1/19/2016 11:10:39 AM The setup batch file on agent machine 'vstfs:///LabManagement/TestMachine/19' exceeded the execution timeout period. vstfs:///LabManagement/TestMachine/19

How to Resolve

To increase this timeout there is no straight forward way using MTM. However update deployment timeout is explained in here, utilizing the tool created by Aseem Bansal. But setting Script timeout not available with this particular tool.

As explained in this MSDN Q&A, I have made changes to the code downloaded from the post here by Aseem Bansal and improved it to enable setting the Script Timeout for test settings. You can download the new utility from Technet Gallery here, and use it as explained below (VS 2013 source code for the tool available here).

UpdateTestSettings /collection:<ProjectCollectionUri> /teamProject:<ProjectName> /settingsname:<TestSettingsName> [/bucketSize:<value>] [/deploymentTimeout:<value>] [/scriptTimeout:<value>]

Examples:

UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProject:myProject /settingsname:My2_0_App /bucketSize:200 /deploymentTimeout:600000

UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProject:myProject /settingsname:My2_0_App /bucketSize:200 /deploymentTimeout:600000 /scriptTimeout:900000

UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProjec:myProject /settingsname:My2_0_App /scriptTimeout:900000

Utility improved to provide Test Settings infor before and after update.

How the code is modified

You can download the original code for the utility from the link provided in the post here.

When this is built with VS 2013 there could be below errors if you do not have VS 2010 in your machine.

With or without having above errors remove the references for TeamFoundation dlls.

There is no Microsoft.TeamFoundation.dll for VS 2013. Microsoft.TeamFoundation.Common.dll would add all thats is added previously by Microsoft.TeamFoundation.dll. Add the references to Team Foundation dlls.

Remove “Read only” from all code files downloaded before change framework.

Set the framework to 4.5 and you can build the code with VS 2013 successflully.

Replace the code in Program.cs with the below updated Program.cs code.

// ---------------------------------------------------------------------------
// <copyright file="Program.cs" company="Microsoft">
//     Copyright (c) Microsoft Corporation. All rights reserved.
// </copyright>
// <summary>
//   Updates the specified properties in the tcm test settings.
// 
//   MTM UI does not expose updation of bunch of test settings properties
//   but the same can be achieved via the TCM APIs. This utility uses those
//   APIs to update the following properties in tcm test settings. 
// 
//   1. Deployment timeout. 
//   2. Bucket size. 
//   3. Apartment state. 
// 
// Introduced ScriptTimeout - Chaminda Chandrasekara 19/01/2016
// </summary>
// ---------------------------------------------------------------------------

using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.TestManagement.Client;
using Microsoft.VisualStudio.TestTools.Common;
using Microsoft.VisualStudio.TestTools.Common.Xml;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Globalization;
using System.Linq;
using System.Reflection;
using System.Text;
using System.Threading;
using System.Xml;

namespace UpdateTestSettings
{
    class Program
    {
        static void Main(string[] args)
        {
            CommandParameters commandParameters;

            Console.WriteLine();
            Console.WriteLine();

            // Validate the parameters
            ValidateArguments(args, out commandParameters);
            Console.WriteLine("Using parameters: {0}", commandParameters);


            // Connect to the team project. 
            TfsTeamProjectCollection collection = new TfsTeamProjectCollection(new Uri(commandParameters.ProjectCollectionUri));
            TestManagementService tcmService = collection.GetService<TestManagementService>();
            ITestManagementTeamProject teamProject = tcmService.GetTeamProject(commandParameters.ProjectName);
            Console.WriteLine("Connected to the team project {0}.", commandParameters.ProjectName);


            // Find the specified test settings.
            TestRunConfiguration testRunConfiguration = null;
            ITestSettings testSetting = FindTestSettings(commandParameters.TestSettingsName, teamProject, out testRunConfiguration);
            
            // Get ScriptTimeout hidden property info
            PropertyInfo pInfo = testRunConfiguration.GetType().GetProperty("ScriptTimeout", BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.DeclaredOnly | BindingFlags.Instance);
            Console.WriteLine("==============================================================");
            Console.WriteLine("Current TestSettings {0} has bucketSize={1}, deploymentTimeout={2}, scriptTimeout={3}  and ApartmentState={4}.", testSetting.Name, testRunConfiguration.BucketSize, testRunConfiguration.DeploymentTimeout, pInfo.GetValue(testRunConfiguration, null), testRunConfiguration.ApartmentState);
            Console.WriteLine("==============================================================");

            Console.WriteLine("Found the testsettings named {0}.", commandParameters.TestSettingsName);

            // If it is an update command, then update the settings
            if (commandParameters.IsUpdate)
            {
                UpdateTestSettings(testSetting, testRunConfiguration, commandParameters);
                Console.WriteLine("Updated the testsettings.");
                Console.WriteLine();
            }

            // Show the new values to the user
            
            // Reload test configurations after update
            FindTestSettings(commandParameters.TestSettingsName, teamProject, out testRunConfiguration);

            Console.WriteLine("==============================================================");
            Console.WriteLine("Updated TestSettings {0} has bucketSize={1}, deploymentTimeout={2}, scriptTimeout={3}  and ApartmentState={4}.", testSetting.Name, testRunConfiguration.BucketSize, testRunConfiguration.DeploymentTimeout, pInfo.GetValue(testRunConfiguration, null),testRunConfiguration.ApartmentState);
            Console.WriteLine("==============================================================");
        }

        /// <summary>
        /// Update the tcm test settings as per the specified command-line parameters
        /// </summary>
        private static void UpdateTestSettings(ITestSettings testSetting, 
                                               TestRunConfiguration testRunConfiguration, 
                                               CommandParameters commandParameters)
        {
            if (commandParameters.HasBucketSize)
            {
                testRunConfiguration.BucketSize = commandParameters.BucketSize;
            }
            if (commandParameters.HasDeploymentTimeout)
            {
                testRunConfiguration.DeploymentTimeout = commandParameters.DeploymentTimeout; 
            }
            if (commandParameters.HasApartmentState)
            {
                testRunConfiguration.ApartmentState = commandParameters.ApartmentState;
            }
            if (commandParameters.HasScriptTimeout)
            {
                PropertyInfo pInfo = testRunConfiguration.GetType().GetProperty("ScriptTimeout", BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.DeclaredOnly | BindingFlags.Instance);

                pInfo.SetValue(testRunConfiguration, commandParameters.ScriptTimeout, null);
            }

            // Tcm test settings contains the TestRunConfiguration xml which has all the settings information. 
            // During the update, we update the testRunConfiguration xml in the test settings
            //
            XmlElement element = LocalXmlPersistence.SaveTestRunConfiguration(testRunConfiguration);
            testSetting.Settings = element;

            testSetting.Save();
        }


        /// <summary>
        /// Find the parameter test settings on the team project
        /// </summary>
        private static ITestSettings FindTestSettings(string testSettingsName, 
                                                      ITestManagementTeamProject teamProject, 
                                                      out TestRunConfiguration testRunConfiguration)
        {
            Console.WriteLine("Finding the testsettings named {0}.", testSettingsName);
            ITestSettings testSetting = null;
            IEnumerable<ITestSettings> testSettings = teamProject.TestSettings.Query("SELECT * FROM TestSettings");

            StringBuilder availableSettings = new StringBuilder();
            foreach (ITestSettings setting in testSettings)
            {
                if (string.Equals(setting.Name, testSettingsName, StringComparison.OrdinalIgnoreCase))
                {
                    if (!setting.IsAutomated)
                    {
                        Console.Error.WriteLine("Warning!! Ignoring testsettings named {0} (with id {1}) as it is a manual testsettings. You should specify only the automated testsettings.", setting.Name, setting.Id);
                        continue;
                    }

                    testSetting = setting;
                    break;
                }

                // We dont want to pick the manual test settings. 
                if (!setting.IsAutomated)
                {
                    continue;
                }
                else
                {
                    if (availableSettings.Length > 0)
                    {
                        availableSettings.Append(",");
                    }

                    availableSettings.Append("\"");
                    availableSettings.Append(setting.Name);
                    availableSettings.Append("\"");
                }
            }

            // Throw error if setting is not found. 
            //
            if (testSetting == null)
            {
                testRunConfiguration = null;

                if (string.IsNullOrEmpty(availableSettings.ToString()))
                {
                    throw new Exception(string.Format(CultureInfo.InvariantCulture, "No automated testsettings found with name {0}. Additionally there is no automated testsettings available in this project. So create an automated testsettings using 'Microsoft Test Manager' and then try again.", testSettingsName));
                }
                else
                {
                    throw new Exception(string.Format(CultureInfo.InvariantCulture, "No automated testsettings found with name {0}. The available automated testsettings are: {1}", testSettingsName, availableSettings));
                }
            }

            // Load the xml from the txm settings into TestRun configuration
            testRunConfiguration = new TestRunConfiguration(testSetting.Name, "dummy");
            testRunConfiguration.Load(testSetting.Settings, XmlTestStoreParameters.GetParameters());
            testRunConfiguration.Name = testSetting.Name;

            return testSetting;
        }


        /// <summary>
        /// Validate the command-line parameters
        /// </summary>
        private static void ValidateArguments(string[] args, out CommandParameters commandParameters)
        {
            if (args.Contains("?", StringComparer.OrdinalIgnoreCase)
                || args.Contains("/?", StringComparer.OrdinalIgnoreCase))
            {
                ShowHelpMessage();
                Environment.Exit(-1);
            }

            string collectionSwitch = "/collection:";
            string projectSwitch = "/teamProject:";
            string testSettingsSwitch = "/settingsname:";
            string bucketSizeSwitch = "/bucketsize:";
            string deploymentTimeoutSwitch = "/deploymentTimeout:";
            string scriptTimeoutSwitch = "/scriptTimeout:";
            string apartmentStateSwitch = "/apartmentState:";
            

            string projectCollectionUri = null;
            string teamProjectName = null;
            string testSettingsName = null;
            int bucketSize = -1;
            int deploymentTimeout = -1;
            int scriptTimeout = -1;
            ApartmentState apartmentState = ApartmentState.Unknown;

            foreach (string argument in args)
            {
                if (argument.StartsWith(collectionSwitch, StringComparison.OrdinalIgnoreCase))
                {
                    projectCollectionUri = argument.Substring(collectionSwitch.Length);
                }
                else if (argument.StartsWith(projectSwitch, StringComparison.OrdinalIgnoreCase))
                {
                    teamProjectName = argument.Substring(projectSwitch.Length);
                }
                else if (argument.StartsWith(testSettingsSwitch, StringComparison.OrdinalIgnoreCase))
                {
                    testSettingsName = argument.Substring(testSettingsSwitch.Length);
                }
                else if (argument.StartsWith(bucketSizeSwitch, StringComparison.OrdinalIgnoreCase))
                {
                    string bucketSizeValue  = argument.Substring(bucketSizeSwitch.Length);
                    bucketSize = int.Parse(bucketSizeValue);
                }
                else if (argument.StartsWith(deploymentTimeoutSwitch, StringComparison.OrdinalIgnoreCase))
                {
                    string deploymentTimeoutValue = argument.Substring(deploymentTimeoutSwitch.Length);
                    deploymentTimeout = int.Parse(deploymentTimeoutValue);
                }
                else if (argument.StartsWith(scriptTimeoutSwitch, StringComparison.OrdinalIgnoreCase))
                {
                    string scriptTimeoutValue = argument.Substring(scriptTimeoutSwitch.Length);
                    scriptTimeout = int.Parse(scriptTimeoutValue);
                }
                else if (argument.StartsWith(apartmentStateSwitch, StringComparison.OrdinalIgnoreCase))
                {
                    string apartmentStateValue = argument.Substring(apartmentStateSwitch.Length);
                    apartmentState = (ApartmentState)Enum.Parse(typeof(ApartmentState), apartmentStateValue, true);
                }
            }

            bool projectCollectionUriError = false, teamProjectError = false, testSettingsError = false;
            if (string.IsNullOrEmpty(projectCollectionUri))
            {
                projectCollectionUriError = true;
            }
            else if (string.IsNullOrEmpty(teamProjectName))
            {
                teamProjectError = true;
            }
            else if (string.IsNullOrEmpty(testSettingsName))
            {
                testSettingsError = true;
            }

            if (projectCollectionUriError || teamProjectError || testSettingsError)
            {
                if (projectCollectionUriError)
                {
                    Console.Error.WriteLine("ProjectCollectionUri argument is not specified");
                }
                else if (teamProjectError)
                {
                    Console.Error.WriteLine("TeamProjectName argument is not specified");
                }
                else if (testSettingsError)
                {
                    Console.Error.WriteLine("Settingsname argument is not specified");
                }

                ShowHelpMessage();
                Environment.Exit(-1);
            }

            commandParameters = new CommandParameters(projectCollectionUri, teamProjectName, testSettingsName);
            commandParameters.BucketSize = bucketSize;
            commandParameters.DeploymentTimeout = deploymentTimeout;
            commandParameters.ScriptTimeout = scriptTimeout;
            commandParameters.ApartmentState = apartmentState;
        }

        private static void ShowHelpMessage()
        {
            Console.Error.WriteLine("Usage: UpdateTestSettings /collection:<ProjectCollectionUri> /teamProject:<ProjectName> /settingsname:<TestSettingsName> [/bucketSize:<value>] [/deploymentTimeout:<value>] [/apartmentState:<value>]");
            Console.Error.WriteLine();
            Console.Error.WriteLine("Examples:");
            Console.Error.WriteLine("UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProject:myProject /settingsname:My2_0_App /bucketSize:200 /deploymentTimeout:600000 /apartmentState:MTA");
            Console.Error.WriteLine();
            Console.Error.WriteLine("UpdateTestSettings /collection:http://abc:8080/tfs/DefaultCollection /teamProject:myProject /settingsname:My2_0_App");
            Console.Error.WriteLine();
            Console.Error.WriteLine("Default values: BucketSize=100 & DeploymentTimeOut=300000 & ApartmentState=STA.");
        }


    }


    /// <summary>
    /// Helper class to capture the command-line inputs
    /// </summary>
    public class CommandParameters
    {
        private string m_projectCollectionUri;
        private string m_teamProjectName;
        private string m_testSettingsName;

        private int m_bucketSize = -1;
        private int m_deploymentTimeout = -1;
        private int m_scriptTimeout = -1;
        private ApartmentState m_apartmentState = ApartmentState.Unknown;


        public CommandParameters(string projectCollectionUri, string teamProjectName, string testSettingsName)
        {
            Debug.Assert(!string.IsNullOrEmpty(projectCollectionUri), "Project collection uri cannot be empty");
            Debug.Assert(!string.IsNullOrEmpty(teamProjectName), "Team Project name cannot be empty");
            Debug.Assert(!string.IsNullOrEmpty(testSettingsName), "TestSettings name cannot be empty");

            m_projectCollectionUri = projectCollectionUri;
            m_teamProjectName = teamProjectName;
            m_testSettingsName = testSettingsName;
        }

        public string ProjectCollectionUri
        {
            get { return m_projectCollectionUri; }
        }

        public string ProjectName
        {
            get { return m_teamProjectName; }
        }
        public string TestSettingsName
        {
            get { return m_testSettingsName; }
        }

        public int BucketSize
        {
            get { return m_bucketSize; }
            set { m_bucketSize = value; }
        }

        public bool HasBucketSize
        {
            get { return m_bucketSize != -1; }
        }

        public int DeploymentTimeout
        {
            get { return m_deploymentTimeout; }
            set { m_deploymentTimeout = value; }
        }

        public bool HasDeploymentTimeout
        {
            get { return m_deploymentTimeout != -1; }
        }

        public int ScriptTimeout
        {
            get { return m_scriptTimeout; }
            set { m_scriptTimeout = value; }
        }

        public bool HasScriptTimeout
        {
            get { return m_scriptTimeout != -1; }
        }

        public ApartmentState ApartmentState
        {
            get { return m_apartmentState; }
            set { m_apartmentState = value; }
        }

        public bool HasApartmentState
        {
            get { return m_apartmentState != ApartmentState.Unknown; }
        }

        public bool IsUpdate
        {
            get { return (HasBucketSize || HasDeploymentTimeout || HasScriptTimeout|| HasApartmentState); }
        }


        public override string ToString()
        {
            StringBuilder builder = new StringBuilder();
            builder.Append(string.Format(CultureInfo.CurrentCulture, "Collection={0},", m_projectCollectionUri));
            builder.Append(string.Format(CultureInfo.CurrentCulture, "TeamProject={0},", m_teamProjectName));
            builder.Append(string.Format(CultureInfo.CurrentCulture, "SettingsName={0}", m_testSettingsName));

            if (HasBucketSize)
            {
                builder.Append(",");
                builder.Append(string.Format(CultureInfo.CurrentCulture, "BucketSize={0},", m_bucketSize));
            }

            if (HasDeploymentTimeout)
            {
                builder.Append(",");
                builder.Append(string.Format(CultureInfo.CurrentCulture, "DeploymentTimeout={0},", m_deploymentTimeout));
            }

            if (HasScriptTimeout)
            {
                builder.Append(",");
                builder.Append(string.Format(CultureInfo.CurrentCulture, "ScriptTimeout={0},", m_scriptTimeout));
            }

            if (HasApartmentState)
            {
                builder.Append(",");
                builder.Append(string.Format(CultureInfo.CurrentCulture, "ApartmentState={0},", m_apartmentState));
            }

            return builder.ToString();
        }
 
    }
}

Chaminda's DevOps Journey with MSFT

Tuesday 19 January 2016

Update Script Timeout in MTM Test Settings

2 comments:

Popular Posts