How the service provider did its Service Desk
Hello! My name is Alexey Volkov. Together with my colleague, developer Alexander Solovyov ( alsov ), we do internal web services in DataLine. This fall, we launched our service desk to replace BMC Remedy. In the post I will tell you why we abandoned the ready-made solution and did everything ourselves.
On average, 450 applications pass through the service desk per day.
I started practicing Remedy almost as soon as I got into the DataLine. After repeated attempts to finish it for our tasks, we decided to make our service desk. Here is a not very short list of reasons why we decided to refuse the BMC Remedy Action Request System:
Inconvenient interface. To register an application, take it to work and mark its permission, we had to make 100,500 clicks.
The page with the incident. Take at least the field for the content of the message that had to be opened in a special window.
A cascade of tabs must be opened to write a solution or redirect the incident to another artist.
Integrationwith other client interfaces and any improvements suggested even those dances with a tambourine. On proprietary, there was almost no room for maneuver. The only loophole was web services that were able to communicate with Remedy, but they were very capricious and unstable. We did some things completely manually and clumsily: I remember how we set up the unloading of reports on incidents with the help of selects from the database.
Of course, there was another way: to hire a contractor and fulfill every whim. There was only one contractor for this software at the time, and he knew about it. Production processes are being adjusted, and improvements would be constantly needed. With this in mind, the price tag turned out inhumane and started from 5 million.
There were not enough licenses.Sometime in the early days of DataLine, we bought 100 licenses. Since then, the company has grown greatly. Licenses were competing. Increasingly, there were situations when the engineer had to wait for the release of the license to start doing his work. Actually, this was the last straw.
Fixing all actions on the incident. We wanted everything related to the technical support of our customers to be reflected in the service desk. Remedy, in fact, only registered requests. All the real work on the incident was carried out in postal correspondence, by telephone, in the smoking room - anywhere, but not in Remedy. Especially if it was a complex application and involved specialists from several departments. The question was solved, but there was no trace of this in Remedy. As a result, the incident was documented by fragments, and sometimes not at all displayed in Remedy. It was very difficult to control, understand and analyze something.
Remedy functions were easy to replace. The most important task of the new service desk is to drag all technical support activities into a “one window”: contacts, work correspondence, documents. We wanted to get some kind of logging of everything that happens after the client sends a request to us for support. Along the way, we wanted to automate many things in order to reduce the load on the first support line and eliminate the human factor to the maximum.
Here are the most important of the functions that we implemented in the new service desk:
Incident transfer.We often receive complex applications that are carried out in a chain by specialists from different departments. From the simplest - an application for physical access to equipment in the data center. First, the dispatcher writes out a pass, then sends the client's data and the pass number to the engineer on duty, who meets and escorts the client to the data center. There are requests that consistently work out 5 departments.
For all such cases, a separate “Submit” button has appeared in the interface.
Correspondence with the client and colleagues. This function is just to ensure that the correspondence on the incident does not "flow away" in the mail and the whole history of communication was preserved with us.
So far everything is very minimal. The plans - to make a full-text editor with the ability to download files, tables, etc.
History of assignments and history of all actions on the incident. These tabs will help to understand controversial issues and internal investigations. At the moment I am uploading reports from the history in order to track how often dispatchers make mistakes when assigning an incident to a profile group.
This is the history of appointments for the internal dismissal of an employee incident.
Incident history Here we will still bring beauty, but the main task has already been solved - everything is logged.
Observer.It happens that in the letter of application the client clips to the copy of the interested parties. All these comrades will appear in the “Observers” tab and monitor the execution of this application. They will receive hits about the status of the request, if they wish, they will be able to correspond with our technical support. Inside the company, this function is also in demand: service managers can fit into any request of their client and monitor its execution.
The list of observers can be edited: delete and add new ones.
Incident TemplatesThere are a lot of typical requests. In order not to fill out an application for skipping or garbage collection dozens of times a day, we added the ability to create incident templates. You just need to click the "Create an incident" button, and the pre-filled template with the specified executor, type, priority and status will open.
There are many regularly recurring requests in the work of data centers: rounds, testings, engineering equipment checks on checklists, etc. For all of this, auto incidents are set up that automatically get into the tracker at a predetermined frequency.
Analytics. Remedy did not have built-in analytics, nothing could be unloaded by regular means. Here we have provided for the unloading of everything and everything for further analysis. For example:
A little later, we want to make dashboards with charts and graphs, so that without any uploads and witchcraft in Excel we will get a picture of what's going on in tech support.
Reports can be generated directly in the service desk using filters.
Uploading data in various formats.
You can select the fields that will be displayed in the upload.
API for integration with other internal services. From simple: service desk pulls up information from directories with a list of client companies, contracts, a list of services ordered and responsible persons who can send inquiries. Previously, you had to first check with a separate file to determine whether the person who wrote the client and can write requests from the company.
“Bypass duty shift” is another of our internal services with which the service desk is now able to interact. This is an electronic check-list, according to which the attendants and operating engineers inspect data centers several times a day for breakdowns and disorder. Situation for example: during the tour, the attendant came across a client rack with incorrectly installed equipment. He simply makes the appropriate note in the “Bypass” program, and an incident on the problem automatically arrives at the service desk. The duty officer proceeds further along the route of the walk, and the problem with the counter is already being solved.
For any of the checklist items, you can create an incident.
The form for the incident inside the software "Bypass". Top hang incidents in the work on the same object.
By the little things.All the most common types of requests we brought to the interface page. After choosing the priority, the user sees the deadline and progress bar showing how much time he has left to execute the incident.
During trial operation, we tested the work of the service desk on internal applications. For about a month, they trained in the divisions of the AHO, capital construction, production and service management. External requests and requests from other departments still went through Remedy.
Then we agreed with three clients to run in the processing of external applications. This is where the first problems happened.
When I first started making a new service desk, I cherished the hope that our smart mail parser would be able to register requests, scatter them across specialized departments, hem messages on the same issue in one incident. In the future, this meant the refusal of dispatchers to process written requests. In Remedy, people are very accustomed to rely on mail correspondence, it was more than we expected. On test tests, the parser failed: he could confuse the departments when assigning a request, he registered new messages on an already open incident as new incidents. There were also more trivial difficulties: the parser could not read the letter that came from the mail with the certificate; did not understand the non-standard coding and sent the text from questions.
It was necessary to add manual labor - Dispatcher appeared. This purgatory for all applications that came to support@dtln.ru . From there, dispatchers manually distribute requests to departments.
On the client side, the logic of interaction with technical support remained the same, only the type of beatings has changed. In the first days there were several overlays with them, mainly due to incorrect contact information in the directories. So one of the clients had 23 responsible persons and there was one general email at all, such as info @. Service Desk obediently all notified by completing the daily rate for incoming on this mailbox in an hour.
In the near future - integration with the Personal Account . All correspondence and the history of calls will be displayed in the client profile. Theoretically, this is a chance to exclude mail from communication with technical support and finally drag the client into our web interface. Let's see how it will take root in life.
The introduction of the parameterized application . There are queries that contain many parameters and values, and it is important not to confuse anything in them. For example, when a client asks to add virtual resources to the pool or create virtual machines of certain sizes. For such cases, we plan to make a constructor with parameters. It will automatically parse similar client requests received by mail. He will be used by dispatchers when accepting applications by telephone.
Here is the parameterized application.
Evaluation of technical support. The notorious "rate our service on a five-point scale" with the ability to write in free form, everything that the client thinks about our service.
We want to develop the same dispatch office, which appeared as a crutch to help the mail parser, into a full-fledged automated dispatcher workstation (AWP) . Now the dispatcher has to look into various interfaces (equipment accounting, list of responsible persons, customer services) in order to collect the necessary information on the customer. In the AWP, everything will be in one window: client incidents, contacts, contracts, ordered services and their parameters, and so on.
Well, do not lose hope for the automatic distribution of applications by department. Now we are thinking in the direction of the hashtag system for the mail parser.
***
In service mode, the service desk has been operating since November, and all this time I am seeing a steady increase in the number of incidents (+ 40%), primarily due to internal requests. I dare to hope that this is due to the fact that the new service desk is more friendly in all respects and requests have ceased to slip past him.
Another profit of the new service desk is flexibility. We have already made several custom solutions for individual customers in order to integrate with their internal service desk or just adapt to their production processes. Previously, it would take months and millions, and now all that is needed is a letter to its developer and 1-2 weeks depending on the complexity of the task.
On average, 450 applications pass through the service desk per day.
What we did not please Remedy?
I started practicing Remedy almost as soon as I got into the DataLine. After repeated attempts to finish it for our tasks, we decided to make our service desk. Here is a not very short list of reasons why we decided to refuse the BMC Remedy Action Request System:
Inconvenient interface. To register an application, take it to work and mark its permission, we had to make 100,500 clicks.
The page with the incident. Take at least the field for the content of the message that had to be opened in a special window.
A cascade of tabs must be opened to write a solution or redirect the incident to another artist.
Integrationwith other client interfaces and any improvements suggested even those dances with a tambourine. On proprietary, there was almost no room for maneuver. The only loophole was web services that were able to communicate with Remedy, but they were very capricious and unstable. We did some things completely manually and clumsily: I remember how we set up the unloading of reports on incidents with the help of selects from the database.
Of course, there was another way: to hire a contractor and fulfill every whim. There was only one contractor for this software at the time, and he knew about it. Production processes are being adjusted, and improvements would be constantly needed. With this in mind, the price tag turned out inhumane and started from 5 million.
There were not enough licenses.Sometime in the early days of DataLine, we bought 100 licenses. Since then, the company has grown greatly. Licenses were competing. Increasingly, there were situations when the engineer had to wait for the release of the license to start doing his work. Actually, this was the last straw.
Fixing all actions on the incident. We wanted everything related to the technical support of our customers to be reflected in the service desk. Remedy, in fact, only registered requests. All the real work on the incident was carried out in postal correspondence, by telephone, in the smoking room - anywhere, but not in Remedy. Especially if it was a complex application and involved specialists from several departments. The question was solved, but there was no trace of this in Remedy. As a result, the incident was documented by fragments, and sometimes not at all displayed in Remedy. It was very difficult to control, understand and analyze something.
Service Desk 2.0
Remedy functions were easy to replace. The most important task of the new service desk is to drag all technical support activities into a “one window”: contacts, work correspondence, documents. We wanted to get some kind of logging of everything that happens after the client sends a request to us for support. Along the way, we wanted to automate many things in order to reduce the load on the first support line and eliminate the human factor to the maximum.
Here are the most important of the functions that we implemented in the new service desk:
Incident transfer.We often receive complex applications that are carried out in a chain by specialists from different departments. From the simplest - an application for physical access to equipment in the data center. First, the dispatcher writes out a pass, then sends the client's data and the pass number to the engineer on duty, who meets and escorts the client to the data center. There are requests that consistently work out 5 departments.
For all such cases, a separate “Submit” button has appeared in the interface.
Correspondence with the client and colleagues. This function is just to ensure that the correspondence on the incident does not "flow away" in the mail and the whole history of communication was preserved with us.
So far everything is very minimal. The plans - to make a full-text editor with the ability to download files, tables, etc.
History of assignments and history of all actions on the incident. These tabs will help to understand controversial issues and internal investigations. At the moment I am uploading reports from the history in order to track how often dispatchers make mistakes when assigning an incident to a profile group.
This is the history of appointments for the internal dismissal of an employee incident.
Incident history Here we will still bring beauty, but the main task has already been solved - everything is logged.
Observer.It happens that in the letter of application the client clips to the copy of the interested parties. All these comrades will appear in the “Observers” tab and monitor the execution of this application. They will receive hits about the status of the request, if they wish, they will be able to correspond with our technical support. Inside the company, this function is also in demand: service managers can fit into any request of their client and monitor its execution.
The list of observers can be edited: delete and add new ones.
Incident TemplatesThere are a lot of typical requests. In order not to fill out an application for skipping or garbage collection dozens of times a day, we added the ability to create incident templates. You just need to click the "Create an incident" button, and the pre-filled template with the specified executor, type, priority and status will open.
There are many regularly recurring requests in the work of data centers: rounds, testings, engineering equipment checks on checklists, etc. For all of this, auto incidents are set up that automatically get into the tracker at a predetermined frequency.
Analytics. Remedy did not have built-in analytics, nothing could be unloaded by regular means. Here we have provided for the unloading of everything and everything for further analysis. For example:
- number of requests per day;
- response speed;
- overdue requests;
- request types;
- loading by department;
- the work of dispatchers;
- the frequency and quality of problems in the context of customers;
- various dependencies, for example, the speed of execution on the type of request.
A little later, we want to make dashboards with charts and graphs, so that without any uploads and witchcraft in Excel we will get a picture of what's going on in tech support.
Reports can be generated directly in the service desk using filters.
Uploading data in various formats.
You can select the fields that will be displayed in the upload.
API for integration with other internal services. From simple: service desk pulls up information from directories with a list of client companies, contracts, a list of services ordered and responsible persons who can send inquiries. Previously, you had to first check with a separate file to determine whether the person who wrote the client and can write requests from the company.
“Bypass duty shift” is another of our internal services with which the service desk is now able to interact. This is an electronic check-list, according to which the attendants and operating engineers inspect data centers several times a day for breakdowns and disorder. Situation for example: during the tour, the attendant came across a client rack with incorrectly installed equipment. He simply makes the appropriate note in the “Bypass” program, and an incident on the problem automatically arrives at the service desk. The duty officer proceeds further along the route of the walk, and the problem with the counter is already being solved.
For any of the checklist items, you can create an incident.
The form for the incident inside the software "Bypass". Top hang incidents in the work on the same object.
By the little things.All the most common types of requests we brought to the interface page. After choosing the priority, the user sees the deadline and progress bar showing how much time he has left to execute the incident.
Implementation
During trial operation, we tested the work of the service desk on internal applications. For about a month, they trained in the divisions of the AHO, capital construction, production and service management. External requests and requests from other departments still went through Remedy.
Then we agreed with three clients to run in the processing of external applications. This is where the first problems happened.
When I first started making a new service desk, I cherished the hope that our smart mail parser would be able to register requests, scatter them across specialized departments, hem messages on the same issue in one incident. In the future, this meant the refusal of dispatchers to process written requests. In Remedy, people are very accustomed to rely on mail correspondence, it was more than we expected. On test tests, the parser failed: he could confuse the departments when assigning a request, he registered new messages on an already open incident as new incidents. There were also more trivial difficulties: the parser could not read the letter that came from the mail with the certificate; did not understand the non-standard coding and sent the text from questions.
It was necessary to add manual labor - Dispatcher appeared. This purgatory for all applications that came to support@dtln.ru . From there, dispatchers manually distribute requests to departments.
On the client side, the logic of interaction with technical support remained the same, only the type of beatings has changed. In the first days there were several overlays with them, mainly due to incorrect contact information in the directories. So one of the clients had 23 responsible persons and there was one general email at all, such as info @. Service Desk obediently all notified by completing the daily rate for incoming on this mailbox in an hour.
What's next?
In the near future - integration with the Personal Account . All correspondence and the history of calls will be displayed in the client profile. Theoretically, this is a chance to exclude mail from communication with technical support and finally drag the client into our web interface. Let's see how it will take root in life.
The introduction of the parameterized application . There are queries that contain many parameters and values, and it is important not to confuse anything in them. For example, when a client asks to add virtual resources to the pool or create virtual machines of certain sizes. For such cases, we plan to make a constructor with parameters. It will automatically parse similar client requests received by mail. He will be used by dispatchers when accepting applications by telephone.
Here is the parameterized application.
Evaluation of technical support. The notorious "rate our service on a five-point scale" with the ability to write in free form, everything that the client thinks about our service.
We want to develop the same dispatch office, which appeared as a crutch to help the mail parser, into a full-fledged automated dispatcher workstation (AWP) . Now the dispatcher has to look into various interfaces (equipment accounting, list of responsible persons, customer services) in order to collect the necessary information on the customer. In the AWP, everything will be in one window: client incidents, contacts, contracts, ordered services and their parameters, and so on.
Well, do not lose hope for the automatic distribution of applications by department. Now we are thinking in the direction of the hashtag system for the mail parser.
***
In service mode, the service desk has been operating since November, and all this time I am seeing a steady increase in the number of incidents (+ 40%), primarily due to internal requests. I dare to hope that this is due to the fact that the new service desk is more friendly in all respects and requests have ceased to slip past him.
Another profit of the new service desk is flexibility. We have already made several custom solutions for individual customers in order to integrate with their internal service desk or just adapt to their production processes. Previously, it would take months and millions, and now all that is needed is a letter to its developer and 1-2 weeks depending on the complexity of the task.