Case Study with Guest OS File Recovery in Veeam Backup & Replication

Original author: Oleg Kulagin
  • Transfer
Hi, Veeam Support Team is with you today. We already told readers of Habr about fantastic creatures of various clients and where they live, and about what and how our department is engaged .

And in the new season, we decided to start publishing technical posts with an analysis of real cases that users are contacting us with. I would like to believe that these materials will help someone understand the intricacies of working with our product without a call to support - and we use the time saved in this way to write new useful articles.

So, today we are disassembling the case “Problem with file level recovery - error while deploying Linux FLR appliance”, which has become one of the most popular over the past months.



Essence of the question


During normal operation, to restore files of a guest OS (not Windows) of a backup virtual machine, the disks of this backup machine are mounted on an auxiliary Linux VM (Linux FLR appliance). After that, you can view the contents of the file system using Veeam Backup Browser, select the necessary files and restore them to the desired location. For more details see here (in English) or here (in Russian).

The secondary VM is temporarily deployed to the ESXi host solely to support recovery, and then retracted. However, when you deploy it in the Veeam Backup & Replication console, you may receive an error message like this:“Linux FLR appliance deploy failed: Module 'MonitorLoop' power on failed.”

How to understand that something went wrong


The nuance is that the problem occurs at a rather specific stage - only when restoring files of a guest OS other than Windows, and specifically when deploying an auxiliary VM.

The error message looks like this in the console:


We see that the problem is with the MonitorLoop module . This is also indicated by the log of the corresponding FLR recovery session, which is stored in a file with the name of the year_month_day_hour_minute_second.log type . In it we find the following entries:

[05.07.2017 17:16:49] <06> Info Mounting restore point. VM: [fileserver], BackupDate: [01/09/2017 18:31:12], Oib: [aa6038d3-bf68-42d6-86c0-de3a48784066]
[07/05/2017 17:17:49] <06> Error Failed to mount oib "Aa6038d3-bf68-42d6-86c0-de3a48784066"
[07/05/2017 17:17:49] <06> Error Linux FLR appliance deploy failed: Module 'MonitorLoop' power on failed. (Veeam.Backup.Common.CAppException)


In addition, since the mount service is responsible for deploying the auxiliary VM (FLR appliance)VeeamMountSvc , then a similar entry will be made in his Svc.VeeamMount log (however, the problem module will not appear in it):

[05.07.2017 17:16:49] <23> Error Recreating WCF proxy ...
[07/05/2017 17 : 16: 49] <23> Error Linux FLR appliance deploy failed (System.ServiceModel.FaultException`1 [Veeam.Backup.Interaction.MountService.CRemoteInvokeExceptionInfo])


"Who's guilty?"


Continuing our investigation, we find out that there is a VMware KB article from which it appears that the MonitorLoop module controls the resources allocated to the virtual machine. Specifically, our error is generated by VMkernel, and it can be found in the VMkernel log:



The root cause is the fact that the ESXi host does not have enough resources to run the secondary VM. Naturally, the process of recovering files without it will fail. To find out what is missing, you can delve into the analysis of VMkernel logs, or you can evaluate the necessary resources based on common sense. And he claims that critical resources are most likely the CPU and RAM available for the VM to work on this host, as well as free space for storing the page file. The disadvantage of the latter is quite common, so if you are sure that the RAM and processor resources are all right, then the cause of the error almost certainly is the lack of space for storing auxiliary VM files and its swap file.

"What to do?"


In order to understand what exactly needs to be fixed, run the File-Level Restore wizard and go to the settings of the auxiliary VM (FLR Helper Appliance).



There are two things to check for the host specified in the Host field :

  1. Does the host have enough memory and CPU resources for the VM to work? The secondary machine consumes a minimum of these resources, so the main thing is that they are available at the time of deployment. If necessary, select another host where these resources are guaranteed to be available.
  2. By default, Veeam saves the paging file of the auxiliary VM to the storage specified as an NFS datastore - this is a regular Windows folder on a mount server. However, this is not always the case.

    The image below shows the ESXi host setting, which is responsible for the default storage location of the paging files for virtual machines: host → Configuration → Virtual Machines → Swap File Location .



    There is a possibility that the default setting - Virtual machine directory (stored in the VM directory) - has been changed, and the place has expired on the newly indicated storage system for this purpose. As a result, it is impossible to deploy new VMs, including auxiliary ones. Check if this is your case.

A similar error can occur with the auxiliary VM during SureBackup - the reason will be the same lack of resources.

Bonus track


But did you know that you can always read more about the work of Veeam products in the online help, which opens by pressing the F1 key from any dialog in the product console, including the main window?

This also applies to the steps of various configuration wizards - press F1 at any step of the wizard, and in your default browser the corresponding paragraph of the documentation opens in the Help Center online help system.

Profit!



We are going to continue to lay out the analysis of popular cases from among those that come to our support. You can express your wishes in the comments. See you soon!

Links to today's post:


• The document "Basic scenarios for using Veeam Backup & Replication 9.5" in Russian
Description of the process of recovering guest OS (FLR) files
VMware knowledge base article

Also popular now: