Outsource IBM Lotus Domino / Notes
This time we decided to diversify our blog and introduce an element of practice. The previously published articles dealt with various technologies, but now it's time to tell what projects we are implementing. The first will be the IBM Lotus Domino / Notes outsourcing project for an infrastructure of one large European financial holding with branches in different parts of the world, which launched in 2010. Our company fulfilled the order for this project, resulting in a successful case in the portfolio. We will discuss further how we do our work and what knowledge and experience in outsourcing IBM Lotus Domino / Notes is ready to apply in the following projects.
Background to the start of the project
The customer has been working with Lotus Domino / Notes since the early 2000s, and was accompanied by a large staff of its own administrators and developers. Within a few years after the introduction of IBM Lotus Domino / Notes, the customer’s IT service refined the standard Lotus Notes applications and developed new ones that automated all the main business processes assigned to Lotus. This idyll lasted until the economic crisis of 2008, which was known to everyone, when financial problems prompted the company’s management to optimize its financial performance by offloading its IT personnel from non-core and routine tasks and concentrating their forces on the main business processes.
As a result, the customer reserved only configuration tasks in the field of information security, audit of the outsourcer’s activities, finalization of the design of mailboxes according to the peculiarities of his business processes, and was responsible for the business logic of the applications. Everything else was outsourced to IT.
First stage. On your marks. What was to be done?
The customer’s IBM Lotus Domino / Notes infrastructure included dozens of servers running mail, Sametime, and applications. Servers and several thousand users were located in locations around the world on several continents. All of this we had to remotely accompany and develop. We had to provide the second and third levels of support, and the customer decided to give the first level (Service Desk) to one of the Eastern European countries, due to the insufficient (at that time) multilingualism of our company’s specialists.
To start the project, only the signed contract was missing, which was implemented by the management of our company and handed the baton to the admins, who were to meet the customer and receive the first portion of the service.
The first stage began withdaily periodic tasks, user and group management, solving incidents of the second level of support and providing the service in 24x7 mode . And six months after the start of the project, we began to provide all types of work specified by the contract, including the third level of support, including communication with the vendor.
Second phase. Project implementation.
Almost immediately after gaining access to the infrastructure of IBM Lotus Domino servers, we found that most of the key business clusters from Domino servers are overloaded and working to the limit. The slightest failure on one of the cluster members led to the fact that user requests were automatically redirected to another server, which also soon became inaccessible from this additional load.
The cluster was falling. And this despite the fact that at the conclusion of the contract, the customer claimed that everything works fine for him and server incidents are extremely rare.
As practice shows, when describing the scale of work transferred to outsourcing a service, customers are prone to downsizing. For example, they indicate only the number of incidents, modestly forgetting about service requests or routine maintenance. And one must be prepared for this. The transition to the virtual environment was in the plans of the customer, but the ever-increasing volumes of information required not only an increase in server performance, but also a change in the architecture of the existing IBM Lotus Domino / Notes infrastructure while ensuring the smooth operation of the customer’s business processes, which we did.
To ensure a high degree of availability, scalability and load balancing, each branch of the customer’s company has one or two dedicated physical servers that host mail, archives and replicas of mission-critical applications. These servers are clustered with the corresponding servers in different data centers. Thus, in case of local problems with the server in the branch, their Notes clients will automatically switch to the server in one of the data centers.
For security, Domino servers are combined in two different Notes domains: “internal” and “external” (mail gateway).
Thanks to optimization, the number of Domino servers has been reduced. Most of them were moved from locations to data centers at the location of the head office. The locations left the minimum number of servers, which allowed the customer to minimize the cost of their support. The mode of operation of users with mailboxes was changed from “On Server” to “On Local”, which led to a significant reduction in the load on mail servers.
The correctness of the concept laid down in the new architecture, which guarantees the continuity of service provision, was also confirmed by regular DR (Disaster Recovery) tests conducted by the customer.
As a result, after a certain adaptation period (we note that, in general, all customer requirements were met in stages within 8 months), the number of server incidents with Domino servers decreased by 2.5 times .
The third stage. Unexpected surprise.
Having dealt with critical server tasks and successfully survived the transition to a new infrastructure in the main locations, we set about proactive work. Under the contract, routine weekly and monthly work passed into our area of responsibility at the very end of the transfer of service to us. This was due to the fact that they contained not only the actions recommended by IBM, but also specific actions aimed at maintaining the operability of business processes and the integrity of customer information.
And here a surprise awaited us. In connection with the optimization of the staff, the remaining administrators of the customer had so much work that they managed to do only critical weekly and monthly tasks. Thus, all the remaining load fell on thefragile shoulders of the outsourcer, that is, on us.
What did we do with a large amount of additional work? It was decided to urgently automate the processes. In the shortest possible time, we wrote scripts for additional analysis of logs, comparison of clusters, quotas for mail files, group analysis, and reporting. All this made it possible to put things in order in 3-4 months and, most importantly, the regular full implementation of all scheduled maintenance led to a reduction not only in the number of server-related, but also in several types of user incidents.
The success of our work was also facilitated by the fact that the customer had mature business processes for administering and resolving incidents, managing and monitoring performance. At first, specialists had to spend a lot of time studying and understanding them, but later on we were able to appreciate their undoubted benefits, creating new ones ourselves and optimizing the old ones.
The results of the project. What did the customer receive?
As a result of the implementation of the project, the specialists of our company provided the customer with a stable, properly developing IT infrastructure and the ability to transfer their own IT resources to solve specialized business problems.
Throughout the project, we:
All this allowed our company to significantly expand the volume of services provided to the customer and after five years to extend the contract.
Background to the start of the project
The customer has been working with Lotus Domino / Notes since the early 2000s, and was accompanied by a large staff of its own administrators and developers. Within a few years after the introduction of IBM Lotus Domino / Notes, the customer’s IT service refined the standard Lotus Notes applications and developed new ones that automated all the main business processes assigned to Lotus. This idyll lasted until the economic crisis of 2008, which was known to everyone, when financial problems prompted the company’s management to optimize its financial performance by offloading its IT personnel from non-core and routine tasks and concentrating their forces on the main business processes.
As a result, the customer reserved only configuration tasks in the field of information security, audit of the outsourcer’s activities, finalization of the design of mailboxes according to the peculiarities of his business processes, and was responsible for the business logic of the applications. Everything else was outsourced to IT.
First stage. On your marks. What was to be done?
The customer’s IBM Lotus Domino / Notes infrastructure included dozens of servers running mail, Sametime, and applications. Servers and several thousand users were located in locations around the world on several continents. All of this we had to remotely accompany and develop. We had to provide the second and third levels of support, and the customer decided to give the first level (Service Desk) to one of the Eastern European countries, due to the insufficient (at that time) multilingualism of our company’s specialists.
To start the project, only the signed contract was missing, which was implemented by the management of our company and handed the baton to the admins, who were to meet the customer and receive the first portion of the service.
The first stage began withdaily periodic tasks, user and group management, solving incidents of the second level of support and providing the service in 24x7 mode . And six months after the start of the project, we began to provide all types of work specified by the contract, including the third level of support, including communication with the vendor.
Second phase. Project implementation.
Almost immediately after gaining access to the infrastructure of IBM Lotus Domino servers, we found that most of the key business clusters from Domino servers are overloaded and working to the limit. The slightest failure on one of the cluster members led to the fact that user requests were automatically redirected to another server, which also soon became inaccessible from this additional load.
The cluster was falling. And this despite the fact that at the conclusion of the contract, the customer claimed that everything works fine for him and server incidents are extremely rare.
As practice shows, when describing the scale of work transferred to outsourcing a service, customers are prone to downsizing. For example, they indicate only the number of incidents, modestly forgetting about service requests or routine maintenance. And one must be prepared for this. The transition to the virtual environment was in the plans of the customer, but the ever-increasing volumes of information required not only an increase in server performance, but also a change in the architecture of the existing IBM Lotus Domino / Notes infrastructure while ensuring the smooth operation of the customer’s business processes, which we did.
To ensure a high degree of availability, scalability and load balancing, each branch of the customer’s company has one or two dedicated physical servers that host mail, archives and replicas of mission-critical applications. These servers are clustered with the corresponding servers in different data centers. Thus, in case of local problems with the server in the branch, their Notes clients will automatically switch to the server in one of the data centers.
For security, Domino servers are combined in two different Notes domains: “internal” and “external” (mail gateway).
Thanks to optimization, the number of Domino servers has been reduced. Most of them were moved from locations to data centers at the location of the head office. The locations left the minimum number of servers, which allowed the customer to minimize the cost of their support. The mode of operation of users with mailboxes was changed from “On Server” to “On Local”, which led to a significant reduction in the load on mail servers.
The correctness of the concept laid down in the new architecture, which guarantees the continuity of service provision, was also confirmed by regular DR (Disaster Recovery) tests conducted by the customer.
As a result, after a certain adaptation period (we note that, in general, all customer requirements were met in stages within 8 months), the number of server incidents with Domino servers decreased by 2.5 times .
The third stage. Unexpected surprise.
Having dealt with critical server tasks and successfully survived the transition to a new infrastructure in the main locations, we set about proactive work. Under the contract, routine weekly and monthly work passed into our area of responsibility at the very end of the transfer of service to us. This was due to the fact that they contained not only the actions recommended by IBM, but also specific actions aimed at maintaining the operability of business processes and the integrity of customer information.
And here a surprise awaited us. In connection with the optimization of the staff, the remaining administrators of the customer had so much work that they managed to do only critical weekly and monthly tasks. Thus, all the remaining load fell on the
What did we do with a large amount of additional work? It was decided to urgently automate the processes. In the shortest possible time, we wrote scripts for additional analysis of logs, comparison of clusters, quotas for mail files, group analysis, and reporting. All this made it possible to put things in order in 3-4 months and, most importantly, the regular full implementation of all scheduled maintenance led to a reduction not only in the number of server-related, but also in several types of user incidents.
The success of our work was also facilitated by the fact that the customer had mature business processes for administering and resolving incidents, managing and monitoring performance. At first, specialists had to spend a lot of time studying and understanding them, but later on we were able to appreciate their undoubted benefits, creating new ones ourselves and optimizing the old ones.
The results of the project. What did the customer receive?
As a result of the implementation of the project, the specialists of our company provided the customer with a stable, properly developing IT infrastructure and the ability to transfer their own IT resources to solve specialized business problems.
Throughout the project, we:
- Provided service with high quality and on time;
- used administration automation tools;
- Used third-party Domino server monitoring software
- built good relations with representatives of the customer;
- prioritized tasks, ways to solve them and the order of interaction with the customer, based on his business processes and mission;
- Worked with the most detailed Service Level Agreement (agreement on the level of service provision);
- constantly improving the service, offering the client new solutions for optimizing IT processes.
All this allowed our company to significantly expand the volume of services provided to the customer and after five years to extend the contract.
Only registered users can participate in the survey. Please come in.
Is your company planning to outsource the IBM Lotus Domino / Notes infrastructure?
- 2.3% Yes, completely the entire infrastructure 2
- 1.1% Yes, but without critical applications 1
- 0% Yes, only mail 0
- 21.1% No, this is not possible 18
- 68.2% Do not use IBM Lotus Domino / Notes 58
- 7% Another answer (indicate in the comments) 6