Building and Deploying Windows Services with VSTS/TFS

Generally windows services are deployed by creating an msi installer. It is possible to deploy msi via VSTS/TFS release management using the extension available in the marketplace. Packaging the windows as non msi would give opportunity to have more control on the configurations etc, when deployment and let’s see how to package a windows service and then get it deployed via VSTS/TFS release management.

Packaging "Assemblies in GAC Installed with SDKs" in Build and Getting Deployed to Target Machine GAC

There can be projects depending on assemblies in Global Assembly Cache installed with may be a internal company SDK, which would even be installed in developer machines and in build servers. These assemblies should be packaged with the project and deployed to the GAC of target machines such as QA,staging and Production etc.  as well. Since the code repo does not include such assemblies in the build server it may be required to extract those assemblies from GAC and packaged with the builds. Let’s see how we can get the assemblies in GAC packaged and get deployed to targets, using VSTS build and release management.

Using NuGet Packages as VSTS Release Artifact Source

If you are used to deploy your solutions with Octopus deploy which a re built with VSTS/TFS, you are used to package your build output as a NuGet package and use it in Octopus. Now you can use the NuGet packages with VSTS release management as well for deployment. For this you have to have the package management feature in VSTS enabled. As VSTS builds and their artifacts are discarded in a configured time period and the maximum time and  number of builds is limited, keeping artifacts as NuGet packages would be useful you to keep your deployed artifacts for a longer period. Let’s explore how to use NuGet packages for deployment in VSTS release management.

VSTS Build, node-sass and HTTP 404

When you are building node js projects with VSTS you may encounter some strange errors which are very hard to diagnose due to misleading error messages. node-sass and Http 404 error is one such error making it really hard to fix the issue misleading error message.

2018-05-28T02:44:19.0012193Z npm info lifecycle node-sass@3.13.1~install: node-sass@3.13.1
2018-05-28T02:44:19.0012339Z Cannot download "https://github.com/sass/node-sass/releases/download/v3.13.1/win32-x64-57_binding.node":
2018-05-28T02:44:19.0012428Z
2018-05-28T02:44:19.0012528Z HTTP error 404 Not Found

2018-05-28T02:44:19.0012596Z
2018-05-28T02:44:19.0012713Z Hint: If github.com is not accessible in your location
2018-05-28T02:44:19.0012897Z       try setting a proxy via HTTP_PROXY, e.g.
2018-05-28T02:44:19.0012971Z
2018-05-28T02:44:19.0013081Z       export HTTP_PROXY=http://example.com:1234
2018-05-28T02:44:19.0013160Z
2018-05-28T02:44:19.0013266Z or configure npm proxy via
2018-05-28T02:44:19.0013335Z
2018-05-28T02:44:19.0013442Z       npm config set proxy http://example.com:8080
2018-05-28T02:44:19.0013573Z npm verb lifecycle node-sass@3.13.1~install: unsafe-perm in lifecycle true
2018-05-28T02:44:19.0014185Z npm verb lifecycle node-sass@3.13.1~install: PATH: C:\Program Files\nodejs\node_modules\npm\bin\node-gyp-bin;F:\agent\_work\32\s\node_modules\gulp-sass\node_modules\node-sass\node_modules\.bin;F:\agent\_work\32\s\node_modules\gulp-sass\node_modules\.bin;F:\agent\_work\32\s\node_modules\.bin;F:\agent\externals\git\cmd;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\ProgramData\chocolatey\bin;C:\Users\svccor_phoenix\AppData\Roaming\npm;c:\tools\nuget;C:\Program Files\dotnet\;F:\agent\Microsoft.Data.Tools.Msbuild\lib\net46;C:\Program Files\Microsoft SQL Server\120\DTS\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\110\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\ManagementStudio\;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn\;C:\Users\svccor_phoenix\AppData\Roaming\npm;c:\tools\ssdtbuild;C:\Program Files\Git\cmd;C:\Program Files\nodejs\;C:\Program Files (x86)\dotnet\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;
2018-05-28T02:44:19.0014766Z npm verb lifecycle node-sass@3.13.1~install: CWD: F:\agent\_work\32\s\node_modules\gulp-sass\node_modules\node-sass

2018-05-28T02:44:19.6231401Z 38604 info lifecycle node-sass@3.13.1~postinstall: Failed to exec postinstall script
2018-05-28T02:44:19.6231571Z 38605 verbose unlock done using C:\Users\svccor_phoenix\AppData\Roaming\npm-cache\_locks\staging-ceadb4797f66e355.lock for F:\agent\_work\32\s\node_modules\.staging
2018-05-28T02:44:19.6231735Z 38606 warn s No repository field.
2018-05-28T02:44:19.6231846Z 38607 warn s No license field.
2018-05-28T02:44:19.6231973Z 38608 verbose stack Error: node-sass@3.13.1 postinstall: `node scripts/build.js`
2018-05-28T02:44:19.6232111Z 38608 verbose stack Exit status 1
2018-05-28T02:44:19.6232266Z 38608 verbose stack     at EventEmitter.<anonymous> (C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\index.js:280:16)
2018-05-28T02:44:19.6232742Z 38608 verbose stack     at emitTwo (events.js:126:13)
2018-05-28T02:44:19.6232869Z 38608 verbose stack     at EventEmitter.emit (events.js:214:7)
2018-05-28T02:44:19.6233040Z 38608 verbose stack     at ChildProcess.<anonymous> (C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\lib\spawn.js:55:14)
2018-05-28T02:44:19.6233187Z 38608 verbose stack     at emitTwo (events.js:126:13)
2018-05-28T02:44:19.6233312Z 38608 verbose stack     at ChildProcess.emit (events.js:214:7)
2018-05-28T02:44:19.6233456Z 38608 verbose stack     at maybeClose (internal/child_process.js:925:16)
2018-05-28T02:44:19.6233598Z 38608 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
2018-05-28T02:44:19.6233728Z 38609 verbose pkgid node-sass@3.13.1
2018-05-28T02:44:19.6233855Z 38610 verbose cwd F:\agent\_work\32\s
2018-05-28T02:44:19.6233975Z 38611 verbose Windows_NT 6.3.9600
2018-05-28T02:44:19.6234132Z 38612 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install"
2018-05-28T02:44:19.6234286Z 38613 verbose node v8.9.3
2018-05-28T02:44:19.6234393Z 38614 verbose npm  v5.5.1
2018-05-28T02:44:19.6234499Z 38615 error code ELIFECYCLE
2018-05-28T02:44:19.6234606Z 38616 error errno 1
2018-05-28T02:44:19.6234736Z 38617 error node-sass@3.13.1 postinstall: `node scripts/build.js`
2018-05-28T02:44:19.6234854Z 38617 error Exit status 1
2018-05-28T02:44:19.6234972Z 38618 error Failed at the node-sass@3.13.1 postinstall script.
2018-05-28T02:44:19.6235125Z 38618 error This is probably not a problem with npm. There is likely additional logging output above.
2018-05-28T02:44:19.6235254Z 38619 verbose exit [ 1, true ]

After investigating the issue for several hours and running the build and trying out options in below links did not help much.

• https://github.com/sass/node-sass/issues/2073
• https://github.com/sass/node-sass/issues/2014
• https://github.com/sass/node-sass/issues/1991
• https://github.com/jamg44/NodeTyped/issues/8

Putting npm install task to provide verbose information helped to identify the possible cause of error which is linked to missing python files.

2018-05-28T02:44:19.0239281Z gyp verb check python checking for Python executable "python2" in the PATH
2018-05-28T02:44:19.0239410Z gyp verb `which` failed Error: not found: python2
2018-05-28T02:44:19.0239557Z gyp verb `which` failed     at getNotFoundError (F:\agent\_work\32\s\node_modules\which\which.js:13:12)
2018-05-28T02:44:19.0239714Z gyp verb `which` failed     at F (F:\agent\_work\32\s\node_modules\which\which.js:68:19)
2018-05-28T02:44:19.0239859Z gyp verb `which` failed     at E (F:\agent\_work\32\s\node_modules\which\which.js:80:29)
2018-05-28T02:44:19.0240001Z gyp verb `which` failed     at F:\agent\_work\32\s\node_modules\which\which.js:89:16
2018-05-28T02:44:19.0240149Z gyp verb `which` failed     at F:\agent\_work\32\s\node_modules\isexe\index.js:42:5
2018-05-28T02:44:19.0240290Z gyp verb `which` failed     at F:\agent\_work\32\s\node_modules\isexe\windows.js:36:5
2018-05-28T02:44:19.0240426Z gyp verb `which` failed     at FSReqWrap.oncomplete (fs.js:152:21)
2018-05-28T02:44:19.0240561Z gyp verb `which` failed  python2 { Error: not found: python2
2018-05-28T02:44:19.0240770Z gyp verb `which` failed     at getNotFoundError (F:\agent\_work\32\s\node_modules\which\which.js:13:12)
2018-05-28T02:44:19.0240927Z gyp verb `which` failed     at F (F:\agent\_work\32\s\node_modules\which\which.js:68:19)
2018-05-28T02:44:19.0241078Z gyp verb `which` failed     at E (F:\agent\_work\32\s\node_modules\which\which.js:80:29)
2018-05-28T02:44:19.0241221Z gyp verb `which` failed     at F:\agent\_work\32\s\node_modules\which\which.js:89:16
2018-05-28T02:44:19.0241362Z gyp verb `which` failed     at F:\agent\_work\32\s\node_modules\isexe\index.js:42:5
2018-05-28T02:44:19.0241508Z gyp verb `which` failed     at F:\agent\_work\32\s\node_modules\isexe\windows.js:36:5
2018-05-28T02:44:19.0241643Z gyp verb `which` failed     at FSReqWrap.oncomplete (fs.js:152:21)
2018-05-28T02:44:19.0241950Z gyp verb `which` failed   stack: 'Error: not found: python2\n    at getNotFoundError (F:\\agent\\_work\\32\\s\\node_modules\\which\\which.js:13:12)\n    at F (F:\\agent\\_work\\32\\s\\node_modules\\which\\which.js:68:19)\n    at E (F:\\agent\\_work\\32\\s\\node_modules\\which\\which.js:80:29)\n    at F:\\agent\\_work\\32\\s\\node_modules\\which\\which.js:89:16\n    at F:\\agent\\_work\\32\\s\\node_modules\\isexe\\index.js:42:5\n    at F:\\agent\\_work\\32\\s\\node_modules\\isexe\\windows.js:36:5\n    at FSReqWrap.oncomplete (fs.js:152:21)',
2018-05-28T02:44:19.0242632Z gyp verb `which` failed   code: 'ENOENT' }
2018-05-28T02:44:19.0242771Z gyp verb check python checking for Python executable "python" in the PATH
2018-05-28T02:44:19.0242910Z gyp verb `which` failed Error: not found: python
2018-05-28T02:44:19.0243049Z gyp verb `which` failed     at getNotFoundError (F:\agent\_work\32\s\node_modules\which\which.js:13:12)
2018-05-28T02:44:19.0243198Z gyp verb `which` failed     at F (F:\agent\_work\32\s\node_modules\which\which.js:68:19)
2018-05-28T02:44:19.0243343Z gyp verb `which` failed     at E (F:\agent\_work\32\s\node_modules\which\which.js:80:29)
2018-05-28T02:44:19.0243500Z gyp verb `which` failed     at F:\agent\_work\32\s\node_modules\which\which.js:89:16
2018-05-28T02:44:19.0243648Z gyp verb `which` failed     at F:\agent\_work\32\s\node_modules\isexe\index.js:42:5
2018-05-28T02:44:19.0243789Z gyp verb `which` failed     at F:\agent\_work\32\s\node_modules\isexe\windows.js:36:5
2018-05-28T02:44:19.0243933Z gyp verb `which` failed     at FSReqWrap.oncomplete (fs.js:152:21)
2018-05-28T02:44:19.0244060Z gyp verb `which` failed  python { Error: not found: python
2018-05-28T02:44:19.0244201Z gyp verb `which` failed     at getNotFoundError (F:\agent\_work\32\s\node_modules\which\which.js:13:12)
2018-05-28T02:44:19.0244356Z gyp verb `which` failed     at F (F:\agent\_work\32\s\node_modules\which\which.js:68:19)
2018-05-28T02:44:19.0244499Z gyp verb `which` failed     at E (F:\agent\_work\32\s\node_modules\which\which.js:80:29)
2018-05-28T02:44:19.0244641Z gyp verb `which` failed     at F:\agent\_work\32\s\node_modules\which\which.js:89:16
2018-05-28T02:44:19.0244796Z gyp verb `which` failed     at F:\agent\_work\32\s\node_modules\isexe\index.js:42:5
2018-05-28T02:44:19.0244944Z gyp verb `which` failed     at F:\agent\_work\32\s\node_modules\isexe\windows.js:36:5
2018-05-28T02:44:19.0245079Z gyp verb `which` failed     at FSReqWrap.oncomplete (fs.js:152:21)
2018-05-28T02:44:19.0245379Z gyp verb `which` failed   stack: 'Error: not found: python\n    at getNotFoundError (F:\\agent\\_work\\32\\s\\node_modules\\which\\which.js:13:12)\n    at F (F:\\agent\\_work\\32\\s\\node_modules\\which\\which.js:68:19)\n    at E (F:\\agent\\_work\\32\\s\\node_modules\\which\\which.js:80:29)\n    at F:\\agent\\_work\\32\\s\\node_modules\\which\\which.js:89:16\n    at F:\\agent\\_work\\32\\s\\node_modules\\isexe\\index.js:42:5\n    at F:\\agent\\_work\\32\\s\\node_modules\\isexe\\windows.js:36:5\n    at FSReqWrap.oncomplete (fs.js:152:21)',
2018-05-28T02:44:19.0245651Z gyp verb `which` failed   code: 'ENOENT' }
2018-05-28T02:44:19.0245779Z gyp verb could not find "python". checking python launcher
2018-05-28T02:44:19.0245997Z gyp verb could not find "python". guessing location
2018-05-28T02:44:19.0246134Z gyp verb ensuring that file exists: C:\Python27\python.exe
2018-05-28T02:44:19.0246273Z gyp ERR! configure error
2018-05-28T02:44:19.0246415Z gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.
2018-05-28T02:44:19.0246574Z gyp ERR! stack     at PythonFinder.failNoPython (F:\agent\_work\32\s\node_modules\node-gyp\lib\configure.js:483:19)
2018-05-28T02:44:19.0246736Z gyp ERR! stack     at PythonFinder.<anonymous> (F:\agent\_work\32\s\node_modules\node-gyp\lib\configure.js:508:16)

Searching along that pointed to the GitHub issue here and using the command below before firing “npm install” in the repo folder using command prompt fixed the issue.

npm --add-python-to-path='true' --debug install --global windows-build-tools

But, when executing with the VSTS the error pops up again due to the user of the VSTS agent is not the same user that executed the above command manually. This could be fixed with the addition of command line step to build which execute the same command before the npm install. Once this additional npm command line “npm --add-python-to-path='true' --debug install --global windows-build-tools” run  it can be disabled from the build definition to save build time, as long as same agent is used for the build for next time.

UPDATE – 24 Oct 2018

If new version of node is installed make sure to run all of below commands before npm install.

npm rebuild node-sass
npm --add-python-to-path='true' --debug install --global windows-build-tools
npm install --global node-gyp

Securing Build Definitions When Multiple Teams Work on a Single Team Project

Securing a build definition is quite straight forward when an organization uses multiple team projects in VSTS/TFS to handle different applications they develop. Each team project build administration can be assigned to different individuals easily. There are organizations using a single team project to manage all of their applications, dividing them into teams inside a single team project. Let's look at possibilities of securing each application teams' builds in single team project for organization scenario.

Visual Source Safe (VSS) to TFS Migration

In modern software development usage of only a source control mechanism would not add value to the software delivery process. However, some organizations still using Visual Source Safe (VSS) to manage their source code, but it is unsupported tool as of now and they should consider moving their code bases to a different tool. As a fully DevOps capable ALM tool TFS is a good choice to move the code from VSS. Let’s look at the steps required to perform such a migration of VSS code, including history.

Finding Active Team Projects Based on Code CheckIns/Commits

When you have many team projects in your TFS instance or in your team services account (VSTS), manually finding out what are the active projects is not going to be an easy task, by looking at each code repository in each of the projects to determine whether project is getting updated with code changes. TFS/VSTS REST API can be used in this scenario to obtain the projects that have any code commits/checkins from a given date using the script available here.

Cloning TFS 2018

How to clone TFS before TSF 2018 is explained in posts, Backing Up and Restoring Databases, Prepare Restored Databases and Configuring the AT. Out of these steps Backing Up and Restoring Databases is valid for TFS 2018. But you do not have to perform the Prepare Restored Databases as it can be done while configuring the Application Tier of the cloned TFS instance. Let’s look at valid steps for cloning TFS 2018.