February 6, 2013 at 15:177 steps to successfully migrate a SharePoint 2007 portal to SharePoint 2010
Hello! This is our first material on Habré. In it, we are going to talk about our experience of migrating a complex portal using the Database Attach method. Enjoy!
Deciding to share our experience of migrating one portal, we focused primarily on those who do not ask “Why migrate?”, But does not ask the question “Maybe right away for 2013?” as well as those who know firsthand the terrible words of the Windows Workflow Foundation, Event Handlers, Jobs, Content Types, Future Receivers, various Site, List, etc. terms and thinks how to make it work in SharePoint 2010.
The article is based on real events, namely the results of many years of cooperation between EastBanc Technologies and a large airline. For five years, we, together with the portal’s IT department, have been working on the Virtual Sales Manager project. This is a non-standard internal corporate portal at MOSS 2007, which automates the routine financial and legal aspects of the sales department and, in fact, is the back office of the department. The portal is used by everyone from directorate to cashiers. It has several products that automate processes such as the conclusion and termination of an agency agreement, registration and cancellation of a point of sale, etc., a huge number of business processes.
Over the five-year history of development, the project has grown into an effective communication system of the company with its agent network, which includes more than 1200 contractors.
So that you can appreciate the scale and scope, we give the numbers and facts:
The development of the portal began in 2007 and continues to this day.
Over the past year, we participated in the migration of several portals from the SharePoint 2007 platform to the SharePoint 2010 platform, in particular, in the migration of Alawar portals and the public reception office of the mayor of Novosibirsk .
Both portals are remarkable for their expanded functionality, the availability of specialized modules (Windows Workflow Foundation, Even Handlers, Jobs), and integration with various BI solutions (Reporting Services, Excel Services, Performance Point).
In Microsoft's promotional products, the migration process looks very simple :
In practice, everything turned out to be not so simple and not so fast: we were faced with the fact that almost all types of “custom” solutions require an individual approach to migration. And if you take into account that, of course, the main thing is to preserve all this non-standard good acquired by overwork and minimize the time for turning off the portal (!), Then the task turns out to be rather ambitious.
The same specificity (only more voluminous) was waiting for us on the migration of the portal for the company, and again we tamed this terrible beast. And then they decided that we already have such experience that may seem interesting and useful to the honorable public.
We began by revising our many years of work related to the development of the VSM portal. In its course, we found out that:
1. In the project, there are 16 types of various custom-extensions of the portal:
2. The project integrates with the following third-party systems:
The migration of each of these types requires its own approach, which we have formed, but more on that later, but first you need:
Here the most interesting part begins: we have a portal in which seemingly compiled source texts are installed, but it does not even open, what do you ask next? About it further.
After the audit, a crucial moment came - it was necessary to choose a migration method. In order to migrate such a complex portal, we decided to create a test site identical to the original portal and gradually revive all the functionality on it. We abandoned the idea of backing up the farm because such an approach would entail a lot of additional work on upgrading the system software of the customer’s farm, and they chose the Database Attach approach. Thus, we saved ourselves the trouble associated with the fact that the new and original infrastructure of the portal did not coincide.
In accordance with this approach, for the full-fledged operation of the test infrastructure on our servers, we configured encrypted tunnels to the customer’s infrastructure, through which we connected:
At this stage, various groups of specialists worked with us, including administrators and the portal security service.
As a result, we received two test sites, each of which consisted of SQL Server 2008 R2 and a web server with SharePoint 2010. At these sites, we deployed original 80-gigabyte content bases. Test sites, as we intended, became an exact copy of the original.
So, we are prepared. We had in our hands a portal identical to the natural one, it was possible to begin testing business processes. X hour was approaching, it was time to take the first step on the path of migration.
Next, we ran pre-upgrade check. Thanks to this, we saw many features necessary for installing. We connected the content base and recompiled all our DLLs under SharePoint 2010.
Te features that were outside the WSP were added to the WSP. We have them in the end we got 16 pieces.
After the next pre-upgrade check, there were much fewer errors, not a single critical one among them.
In the next step, we added the * .config, * .sitemap files to configure the workflow and outgoing mails from the folder “C: \ inetpub \ wwwroot \ wss \ VirtualDirectories \ 80” to the root of the site. The portal opened, and we were able to see what was supposed to migrate on its own (document libraries, site structure, lists), but, unfortunately, we could only see it.
Having seen the portal, we wrote out a check list of what does not work. It was necessary to make it work. You can imagine that the checklist was not small, we took every role and compiled a list of available actions, for example, “anonymous user”, “cashier”, “authorized agent”, etc.
Primus needed to be fixed:
The same was with the task list for Workflow.
Finally, earned all the sheets.
And then for two weeks we walked around the site, found out that it was crashing and repaired all the custom solutions that required it.
For example, it was required to fix:
When it seemed to us that everything was ready, we deployed 2 virtual machines with the old airline base and went all the way again (in the checklist we described 16 steps that needed to be done, we repeated them). Everything was restored in the state in which it was required. It was possible to prepare the infrastructure and ... migration!
When preparing the infrastructure, it was advisable for us to keep the old portal in order to return to it in case of failure. Therefore, a backup of the old infrastructure was made, which took about 4 hours.
The logic of the task was also added by the fact that we did not have direct access to the infrastructure, and the preparations took place at night, when we did not want to disturb the technical service in case of problems. Therefore, they resorted to using the ILO remote access system, with which they installed the operating system.
On Friday, at 9 o’clock in the evening in Moscow, we turned off the server, made a backup, killed all the systems and turned on the new ones for installation. Here the main adventures began: instead of an hour, the system was set through ILO for three hours! And it took us 7 hours to install three systems, whereas initially we planned to spend no more than two! With the motto "Do not retreat and do not give up" on our lips we spent a sleepless Friday night.
When, finally, on Saturday at 6 a.m. everything was installed, and we had normal access through the remoot desktop, we installed SharePoint 2010 and SQL on the systems, deployed the database, tested it (see Step Five) and connected. Already on Saturday after lunch (12 hours after the start of the epic), we had a 90% working portal.
So to summarize.
We spent 4 weeks preparing the migration, for which everything was done to minimize downtime: a test site was created that was identical to the original portal; with the help of this platform, a “training” was conducted - test migration; Based on the test results, a detailed plan of the “combat” migration was compiled.
The preparations worked - the migration passed with a minimum portal shutdown time! Namely, the portal did not work for 17 hours, including a 12-hour rearrangement of operating systems and hardware settings. All this happened on the weekend and almost did not affect the work of agents. On Monday, the portal operated as usual, we won!
Thank you for mastering this epic work. We will be happy to talk about migration, answer questions and argue (constructively!).
A little about who is likely to be interested in reading this.
Deciding to share our experience of migrating one portal, we focused primarily on those who do not ask “Why migrate?”, But does not ask the question “Maybe right away for 2013?” as well as those who know firsthand the terrible words of the Windows Workflow Foundation, Event Handlers, Jobs, Content Types, Future Receivers, various Site, List, etc. terms and thinks how to make it work in SharePoint 2010.
Portal History
The article is based on real events, namely the results of many years of cooperation between EastBanc Technologies and a large airline. For five years, we, together with the portal’s IT department, have been working on the Virtual Sales Manager project. This is a non-standard internal corporate portal at MOSS 2007, which automates the routine financial and legal aspects of the sales department and, in fact, is the back office of the department. The portal is used by everyone from directorate to cashiers. It has several products that automate processes such as the conclusion and termination of an agency agreement, registration and cancellation of a point of sale, etc., a huge number of business processes.
Over the five-year history of development, the project has grown into an effective communication system of the company with its agent network, which includes more than 1200 contractors.
So that you can appreciate the scale and scope, we give the numbers and facts:
The development of the portal began in 2007 and continues to this day.
- Over 35,000 man-hours have been spent on its creation.
- At different times, 28 people worked on the project.
- More than 1600 active users work in the system daily.
- The size of the portal content database is 80 GB.
- Geography of users - the whole Russian Federation, CIS, Southeast Asia, Europe.
- The system works in all time zones.
Why did we decide to write about it
Over the past year, we participated in the migration of several portals from the SharePoint 2007 platform to the SharePoint 2010 platform, in particular, in the migration of Alawar portals and the public reception office of the mayor of Novosibirsk .
Both portals are remarkable for their expanded functionality, the availability of specialized modules (Windows Workflow Foundation, Even Handlers, Jobs), and integration with various BI solutions (Reporting Services, Excel Services, Performance Point).
In Microsoft's promotional products, the migration process looks very simple :
- We carry out infrastructure planning.
- Install the portal and configure it.
- We choose the method of updating the portal to 2010.
- Updated and done!
In practice, everything turned out to be not so simple and not so fast: we were faced with the fact that almost all types of “custom” solutions require an individual approach to migration. And if you take into account that, of course, the main thing is to preserve all this non-standard good acquired by overwork and minimize the time for turning off the portal (!), Then the task turns out to be rather ambitious.
The same specificity (only more voluminous) was waiting for us on the migration of the portal for the company, and again we tamed this terrible beast. And then they decided that we already have such experience that may seem interesting and useful to the honorable public.
Audit or inventory accounting
We began by revising our many years of work related to the development of the VSM portal. In its course, we found out that:
1. In the project, there are 16 types of various custom-extensions of the portal:
- a. Windows Workflow Foundation - 21 business processes were launched (for example, such as canceling a point of sale, entering volumes of flights under a corporate agreement, a question from the user, etc.). Some processes are running in the amount of 3,000 instances.
- List Templates - 232 pieces
- Web-sites' templates
- Main portal
- News feed
- BSP Agent Portal
- Cashier Bonus Program Portal
- Confidential Tariffs Node
- Agent Assistance Node (Agent Requests)
- More than 66 Custom Event Receivers and Feature Receivers
- Designs
- For the main site
- For Agent Inquiries
- For BSP
- For Cashier Bonus
- Jobs (pieces 15)
- Computed fields (about 80)
- Custom Actions with custom controls (about 30).
2. The project integrates with the following third-party systems:
- counterparty accounting system;
- ASIA NEXT software - ordering stock tickets, claim work.
The migration of each of these types requires its own approach, which we have formed, but more on that later, but first you need:
- Collect all the sources in one heap and create a new project in VS 2010, place them with TFS.
- Make everything assembled (as it turned out, it’s not so difficult).
- Create a WSP to roll onto the portal.
- Complete the pre-upgrade check procedure.
Here the most interesting part begins: we have a portal in which seemingly compiled source texts are installed, but it does not even open, what do you ask next? About it further.
Infrastructure and the approach to migration
After the audit, a crucial moment came - it was necessary to choose a migration method. In order to migrate such a complex portal, we decided to create a test site identical to the original portal and gradually revive all the functionality on it. We abandoned the idea of backing up the farm because such an approach would entail a lot of additional work on upgrading the system software of the customer’s farm, and they chose the Database Attach approach. Thus, we saved ourselves the trouble associated with the fact that the new and original infrastructure of the portal did not coincide.
In accordance with this approach, for the full-fledged operation of the test infrastructure on our servers, we configured encrypted tunnels to the customer’s infrastructure, through which we connected:
- Active Directory
- Web Services for Integration (Clause 3 of the previous section)
- Oracle database.
At this stage, various groups of specialists worked with us, including administrators and the portal security service.
As a result, we received two test sites, each of which consisted of SQL Server 2008 R2 and a web server with SharePoint 2010. At these sites, we deployed original 80-gigabyte content bases. Test sites, as we intended, became an exact copy of the original.
So, we are prepared. We had in our hands a portal identical to the natural one, it was possible to begin testing business processes. X hour was approaching, it was time to take the first step on the path of migration.
First step
Next, we ran pre-upgrade check. Thanks to this, we saw many features necessary for installing. We connected the content base and recompiled all our DLLs under SharePoint 2010.
Te features that were outside the WSP were added to the WSP. We have them in the end we got 16 pieces.
After the next pre-upgrade check, there were much fewer errors, not a single critical one among them.
In the next step, we added the * .config, * .sitemap files to configure the workflow and outgoing mails from the folder “C: \ inetpub \ wwwroot \ wss \ VirtualDirectories \ 80” to the root of the site. The portal opened, and we were able to see what was supposed to migrate on its own (document libraries, site structure, lists), but, unfortunately, we could only see it.
Second step
Having seen the portal, we wrote out a check list of what does not work. It was necessary to make it work. You can imagine that the checklist was not small, we took every role and compiled a list of available actions, for example, “anonymous user”, “cashier”, “authorized agent”, etc.
Third step
Primus needed to be fixed:
- Custom-fields (caught 45 pieces), which for some reason did not want to display correctly. We decided that we would repair them as recommended in the manuals for SharePoint 2010, that is, through the XSLT transformation. It worked!
- Lots of custom Workflows written in Visual Studio and custom pages that used Ajax. Previously, when switching from page to page, we transmitted ViewState. After migrating to SharePoint 2010, it ceased to be transferred. For a long time we tried to understand - why? It turned out that Java scripts changed in SharePoint 2010 and therefore, when entering a new page, the form changed its action path. We solved this problem as follows: on each such page we wrote our additional script, which forbade changing this custom path. And it all worked!
- A lot of Windows Workflow Fondation with many different versions of the DLL, which were gradually added all these years, and as a result, on the eve of the migration, some Workflows had 3-4 versions of the DLL. In order not to transfer all these DLLs, but to use the functionality from the new DLLs, we first recompiled them under SharePoint 2010, and secondly, we specified binders for these DLLs in the configurations. As a result, when the system requested a DLL of one version, it was always given the DLL of another version, which we defined. After that, many problems were also avoided: working Workflows continued to work, and new ones started as needed, with the correct versions.
- Lists of ASPX pages. After migration, they began to work very slowly. Here we are at a standstill for a while. As a result, we decided to reconsider queries to the SharePoint database with the SQL profiler and find out what goes into the database. It turned out that SharePoint is trying to stretch all ASPX pages at a time. They tried to create a new list and wrote a script that copied the list pages from the old sheet to the new one. Everything worked quickly on it. After that, we wrote a script that returned all ASPX pages in the old list to the original version of the site - everything was repaired.
- Large lists of points of sale. SharePoint granted rights to each element individually. There were about 15 thousand points of sale, for which the rights were distributed separately, so the lists were terribly slow. We managed it like this: for each agent we got a folder, we distributed rights to this folder and dragged agent's sales points there. We reduced the number of rights to the number of agents, and the list began to work much faster.
The same was with the task list for Workflow.
Finally, earned all the sheets.
Fourth step (boring routine)
And then for two weeks we walked around the site, found out that it was crashing and repaired all the custom solutions that required it.
For example, it was required to fix:
- Translating list views into XSL. We did not translate them specially, only a few fields. The rest of the lists we have earned as it should.
- Job and Handlers. I had to redefine the property “HasAdditionalUpdateAccess” in the job class to make it possible to create a job from the site’s UI.
- JQuery widgets. Previously, the $ .ajax method simply returned an object, but now it began to add it inside the “d” object (there was data - it became data.d).
Fifth Step (training)
When it seemed to us that everything was ready, we deployed 2 virtual machines with the old airline base and went all the way again (in the checklist we described 16 steps that needed to be done, we repeated them). Everything was restored in the state in which it was required. It was possible to prepare the infrastructure and ... migration!
The sixth step (we are preparing the customer’s infrastructure)
When preparing the infrastructure, it was advisable for us to keep the old portal in order to return to it in case of failure. Therefore, a backup of the old infrastructure was made, which took about 4 hours.
The logic of the task was also added by the fact that we did not have direct access to the infrastructure, and the preparations took place at night, when we did not want to disturb the technical service in case of problems. Therefore, they resorted to using the ILO remote access system, with which they installed the operating system.
On Friday, at 9 o’clock in the evening in Moscow, we turned off the server, made a backup, killed all the systems and turned on the new ones for installation. Here the main adventures began: instead of an hour, the system was set through ILO for three hours! And it took us 7 hours to install three systems, whereas initially we planned to spend no more than two! With the motto "Do not retreat and do not give up" on our lips we spent a sleepless Friday night.
Seventh step (we clicked on the switch!)
When, finally, on Saturday at 6 a.m. everything was installed, and we had normal access through the remoot desktop, we installed SharePoint 2010 and SQL on the systems, deployed the database, tested it (see Step Five) and connected. Already on Saturday after lunch (12 hours after the start of the epic), we had a 90% working portal.
And there was happiness to all
So to summarize.
We spent 4 weeks preparing the migration, for which everything was done to minimize downtime: a test site was created that was identical to the original portal; with the help of this platform, a “training” was conducted - test migration; Based on the test results, a detailed plan of the “combat” migration was compiled.
The preparations worked - the migration passed with a minimum portal shutdown time! Namely, the portal did not work for 17 hours, including a 12-hour rearrangement of operating systems and hardware settings. All this happened on the weekend and almost did not affect the work of agents. On Monday, the portal operated as usual, we won!
Thank you for mastering this epic work. We will be happy to talk about migration, answer questions and argue (constructively!).