It is great if the latest updates to TFS can be applied as and when they are released. But for a large organization it might not be sometimes easy. There may be few version gaps when you try to upgrade your TFS. Let’s discuss a problem with SQL server, faced while upgrading from TFS 2015.2 to TFS 2018, and how can it be fixed without getting into deeper troubles. This upgrade was done after a pre production trial using a clone of the TFS 2015.2, and no issues faced during TFS2015.2 from to TFS 2018 in the trial steps, which were exactly followed in production, except for clone TFS 2015.2 of course. Still in a production scenario you might run into unexpected 
The Problem
The production TFS 2015.2 environment is having an AT and a separate DB server containing TFS DBs and the reporting services. TFS services were stopped using TFSServiceControl quiesce command. The DBs were backed up. Since TFS 2018 only supporting SQL 2016 SP1 as minimum SQL server version SQL upgrade was required. As done in the trial SQL Server was upgraded from SQL 2012 SP1 CU2, to SQL 2012 SP2 to allow SQL 2016 upgrade. SQL 2012 SP2 update was successful and no issues reported. Restarted TFS services with TFSServiceControl unquiesce and verified TFS is up and running. The stopped the TFS services again using TFSServiceControl quiesce and SQL 2016 SP1 upgrade started. At the last stage of the SQL upgrade an error reported as shown below.
TITLE: Microsoft SQL Server 2016 Setup
------------------------------
The following error has occurred:
Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
------------------------------
BUTTONS:
OK
------------------------------
Once click OK in this message installation completed after few seconds with all other services installed successfully except reporting services. Since link in the error message was not useful at all, searching the error message pointed to a forum thread describing the issue https://social.msdn.microsoft.com/Forums/en-US/62193bc5-55b8-4793-9ac3-891080b4a5cb/upgrading-ssrs-2014-to-2016-index-was-out-of-range-must-be-nonnegative-and-less-than-the-size-of?forum=sqlsetupandupgrade. As it is mentioned there was SSL bindings available in the reporting services and option in the thread asked revert back to original state of the VM and start upgrade after removing SSL binding from reporting services. Time for panic, since this is production upgrade, and this server cannot be rolled back to previous state as no one taking snapshots of it and I was not even sure whether this DB server is a VM or a real physical server. Uninstall SQL 2016 and install SQL 2012 SP2 and restore TFS DBs from backup and start from beginning was one choice. But it was not simple as TFS DBs are huge size and would take several hours to restore the DBs extending the downtime of the TFS for entire team. Lucky it was long weekend with a Christmas holiday
The solution.
Run the remove from control panel only for SQL reporting services, and uninstall the corrupted upgrade. This is not supposed to remove ReportServer or ReportServerTemp database. Even if they are removed restoring them from backups will not take long time.
Once the reporting services removed successfully, installation of reporting services performed for same server instance and it does not allow you to configure as it installs.
Configured the reporting services using the existing reporting databases, and restored the reporting services key. Verified and the reporting services were up and running smoothly.
Saved about 10 –12 hours of DB restore task, and completed the rest of the TFS upgrade as easy as it could be. TFS Search services for work item search and code search was setup in a separate server from TFS AT and let’s talk about it in the next post.
.
No comments:
Post a Comment