Running UI Tests with Unified Agent

You needed to setup dedicated machines for testing when using Run Functional Tests and you had to execute, Deploy Test Agent step and Copy Files step as prerequisites. With the introduction of unified agent enabling build, deployment and testing using same agent, now it is possible to setup pool of machines as Test clients to cater multiple project needs. You can even deploy and test using the same agent if you wish to do so.  Unified agent support is available to run tests as multi-machine distributed execution. You do not require to execute copy files task as artifacts downloaded by the agent can be used for running tests. Let’s look at how to prepare a unified agent for running UI tests.

You have to download Visual Studio Team Services or TFS build/deploy/test agent. You need to create a Personal Access Token (PAT) to enable configuration of the agent. Once you download the agent and extract the files to a local folder in the required machine, you can run config.cmd to configure the agent. You have to specify the agent to run in interactive mode if you intend to run unit tests. Auto logon with screen saver disabled can be configured to enable a user account to automatically login and execute UI tests.

Once configuration completes for the restart later question you have to provide N to say no. Then the agent machine will restart and would be in a mode ready to execute tests.

You do not have to log onto the machine. if you restart the machine it will auto logon and will be ready for test execution. In your release management environment you can use version 2, of Visual Studio Test task to enable running tests on agent machine. If you are running UI tests you have to select “Test mix contains UI tests” option to notify you are requiring interactive agents. If your agent pool contains agents with interactive mode and agents run as service, selecting this option will use only the interactive agents for running the tests, when you are running with multiple agent mode in an agent phase.

Set the control options of the task, to continue on error will make the environment to partially succeed even if running tests fail.

The test UI test can be executed and it is much less time consuming as it does not require the overhead of deploy test agent etc.

There are limitation of using the Visual Studio Test task version 2 as of now. It requires to have relevant Visual Studio version (2015/2017) installed on the agent running the test. The hosted agents in Visual Studio Team Service does not support execution of UI tests with the task as the agent are running as service in them. Hope these limitations will be improved sooner.