Backup for Linux does not write letters
Hello!
Today I want to talk about how to manage Veeam Agent for Linux using the command line, and what opportunities it opens in the capable hands of a programmer.
Writing an article pushed me to comment on the previous article . I paraphrase the user's surprise: “Well, how is that? The server does not write letters that he was crap! ”. Moreover, according to analysts, he is not the only one, otherwise the thread would not have appeared on the forum. And since people write, it means that someone needs it!
In the article I will explain why this function is not in the product. But we will not stop at this, we will add this feature! We are programmers, so we will write a letter and generate a report in the form of an html page.
In addition, I will show the most useful, in my opinion, commands that can facilitate the work of the administrator.
Get ready: a lot of code, no pictures.
To begin with, we will answer the question: “Why doesn't Veeam Agent for Linux write letters?”
You may not like the answers, do not judge me. But the fact is that more or less large enterprise users do not need it, and here's why:
In the version of Veeam Backup & Replication 9.5 Update 4 , it is possible to use this product for free, but with restrictions on the serviced virtual / physical machines.
If you have up to 3 (inclusive) physical servers - free VBR functions will be more than enough for you.
If you have more than 3 machines, you don’t have the opportunity to pay for the software, and you still want to centrally monitor your servers, then I suggest you add some scripts yourself. I like to amuse myself in python after working for C / C ++.
Scripts, we will wrap the call command
The command description for Veeam Agent for Linux is valid for version 3.0. On previous versions, I did not check, so there may be differences.
The CLI interface in Veeam Agent for Linux is quite convenient and well documented. Just enter
In order to see what each command allows you to do, just call
Here, by the way, we can see the log collection command
There is another interesting team that appeared in version 3.0:
The fact is that starting with version 3.0, the user is required to explicitly agree with the license agreements. It looks like this:
Accordingly, the work of your scripts may be disrupted. In order not to enter each machine and not to carry out this procedure manually, the following commands were provided:
They allow you to accept license agreements without any questions.
But we deviated from the subject of writing a letter.
For the task of monitoring the status of the server, we need a command
Well, there is information here, when the server backed up and what was the success. In principle, it is already possible to collect the “exhaust” to a file and send it by letter. However, over the course of a year, the letter may grow by about 365 lines. And looking into the text of the State with errors may seem tedious. Therefore, this “exhaust” is parsed and we get a normal list with which you can do something already.
View full code here.
Now let's make a letter and send it to ourselves.
As a result, after installing mailutils, we can receive a letter of the form:
The letter displays only the last 10 sessions. In this case, at the very beginning of the letter, information about the number of successful and not-so sessions is displayed. Just look at the tsiferki in the letter at the beginning of the working day, checking mail and sipping coffee to understand that the night backups were successful.
If you need something clearer - you can request information about the sessions in xml-format and transfer it to your server. They combine the data into a single summary table that displays all the necessary information in a convenient or convenient format for you.
XML get a couple of lines:
Save the resulting file
Next, send the resulting XML to the server. An alternative option is also possible - the server collects XML from the machines that are backed up. Who is the initiator is not important for us. It is important that XML servers with session lists from all machines are collected on the server. I chose the first option:
Now on the server side it remains to process the received data and make a beautiful html-page.
As a result, the report is ready:
I did not have the task to make a beautiful product. The task was to show that the issue of collecting statistics can be solved on scripts in one or two days.
In principle, if you develop the ideas outlined here, then you can do "Open Backup Monitor for Veeam Agent for Linux". In my opinion, a good topic for coursework in Python, or, maybe, even for a diploma, or just a reason to practice programming in an opensource project. Agree, it is better to practice programming, than to become an 80th-level elf.
All code can be found at http://www.github.com/CodeImp/veeampy/ . Download, use, supplement and fork for health.
Note, the code is distributed under the GPL-2 license, it may contain errors and so on. Everything is as usual in the world of opensource. So before you apply in production - do not forget to drive on the test lab.
Today I want to talk about how to manage Veeam Agent for Linux using the command line, and what opportunities it opens in the capable hands of a programmer.
Writing an article pushed me to comment on the previous article . I paraphrase the user's surprise: “Well, how is that? The server does not write letters that he was crap! ”. Moreover, according to analysts, he is not the only one, otherwise the thread would not have appeared on the forum. And since people write, it means that someone needs it!
In the article I will explain why this function is not in the product. But we will not stop at this, we will add this feature! We are programmers, so we will write a letter and generate a report in the form of an html page.
In addition, I will show the most useful, in my opinion, commands that can facilitate the work of the administrator.
Get ready: a lot of code, no pictures.
To begin with, we will answer the question: “Why doesn't Veeam Agent for Linux write letters?”
You may not like the answers, do not judge me. But the fact is that more or less large enterprise users do not need it, and here's why:
- First, to work with mail, you must either put the smpt-server on the local machine, or use some kind of inside the network. At the simplest implementation (command
mail), you will need to install the mailutils package . And many system administrators will not want to create a potential vulnerability in their production server in the form of a service that can send letters anywhere. Yes, and the opportunity may not be due to the closeness of the ports, the independence of subnets and so on ... - Secondly, since the mailutils package may not be very much on the system (for the first reason), there is no point in trying to use it. Otherwise, we can get a function that seems to be there, but “out of the box” does not work, which means that there will be a thread on the forum on the topic of the type: “How to set up the server so that letters are sent out.”
- Well and thirdly, in general, no additional notification is needed, since more or less large enterprise customers use Veeam Backup & Replication . His console collects information about all backups that have been made to known repositories. See for yourself.
In the version of Veeam Backup & Replication 9.5 Update 4 , it is possible to use this product for free, but with restrictions on the serviced virtual / physical machines.
If you have up to 3 (inclusive) physical servers - free VBR functions will be more than enough for you.
If you have more than 3 machines, you don’t have the opportunity to pay for the software, and you still want to centrally monitor your servers, then I suggest you add some scripts yourself. I like to amuse myself in python after working for C / C ++.
Scripts, we will wrap the call command
veeamconfig. The team veeamconfigprovides access to all the functionality of the product. Undoubtedly, a pseudographic interface created using the ncurses library, much more pleasant for the eyes, but if you want to link the program into something new, then the CLI is our everything. The command description for Veeam Agent for Linux is valid for version 3.0. On previous versions, I did not check, so there may be differences.
The CLI interface in Veeam Agent for Linux is quite convenient and well documented. Just enter
veeamconfig --helpand you will get a list of available commands:$sudo veeamconfig --help
Veeam Agent for Linux
(c) Veeam Software AG
Usage: veeamconfig [command]
Commands:
repository - Backup repositories management
vbrserver - Veeam Backup and Replication servers management
job - Backup jobs management
backup - Backups management
point - Restore points management
license - License management
agreement - End User License Agreement management
config - Import/export configuration
schedule - Jobs schedule configuration
cloud - Cloud provider management
mode - Operation mode
session - Sessions management
ui - User interface
aap - Application-aware processing
version, --version, -v - Product version
help, --help, -h - Short help
In order to see what each command allows you to do, just call
veeamconfig config --help. We get:Veeam Agent for Linux
(c) Veeam Software AG
Usage: veeamconfig config [command]
Commands:
import - Import repositories and jobs into database
export - Export repositories and jobs from database
grabLogs - Collect support logs bundle
patchiso - Create custom Veeam Recovery Media adding all hardware drivers from this system
help, --help, -h - Short help
Here, by the way, we can see the log collection command
grabLogs. It allows you to quickly collect all the necessary logs for support. This is in case something goes wrong. There is another interesting team that appeared in version 3.0:
$ sudo veeamconfig agreement --help
Veeam Agent for Linux
(c) Veeam Software AG
Usage: veeamconfig agreement [command]
Commands:
acceptEula - Accept Veeam End User License Agreements
acceptThirdPartyLicenses - Accept Veeam 3rd party License Agreement
show - Show End User License Agreements acceptance status
help, --help, -h - Short help
The fact is that starting with version 3.0, the user is required to explicitly agree with the license agreements. It looks like this:
$ sudo veeamconfig job list
I accept Veeam Software End User License Agreement:
/usr/share/doc/veeam/EULA
(yes/no | y/n):
yes
I accept the terms of the following 3rd party software components license agreements:
/usr/share/doc/veeam/3rd_party
(yes/no | y/n):
Accordingly, the work of your scripts may be disrupted. In order not to enter each machine and not to carry out this procedure manually, the following commands were provided:
veeamconfig agreement acceptEula
veeamconfig agreement acceptThirdPartyLicenses
They allow you to accept license agreements without any questions.
But we deviated from the subject of writing a letter.
For the task of monitoring the status of the server, we need a command
veeamconfig session list. It displays something like:Job name Type ID State Started at Finished at
bj-home Backup {dbe48e88-3df7-4712-a472-09af8fed4e80} Success 2018-12-05 15:43 2018-12-05 15:44
bj-home Backup {c178a799-2935-4bd6-883b-b11278000076} Success 2018-12-05 16:26 2018-12-05 16:26
bj-home Backup {3405dad3-0016-4a00-933e-60ef66b30324} Success 2018-12-06 06:00 2018-12-06 06:00
Well, there is information here, when the server backed up and what was the success. In principle, it is already possible to collect the “exhaust” to a file and send it by letter. However, over the course of a year, the letter may grow by about 365 lines. And looking into the text of the State with errors may seem tedious. Therefore, this “exhaust” is parsed and we get a normal list with which you can do something already.
View full code here.
classCSession: @staticmethoddefList():return subproccall( ["veeamconfig", "session", "list"] )
classCSessionInfoList(object):def__init__(self, list):
self.list = list
defList(self):return self.list
@staticmethoddefGet():
text = CSession.List()
lines = text.split("\n")
list = [] # session info listfor line in lines:
if len(line) == 0:
continue
words = line.split()
if len(words) == 0:
continueif words[0] == "Job":
continueif words[0] == "Total":
continuetry:
jobName = words[0]
type = words[1]
id = words[2]
state = words[3]
startTime = words[4] + " " + words[5]
finishTime = words[6] + " " + words[7]
list.append(CSessionInfo(id, type, jobName, state, startTime, finishTime))
except:
print"Failed to parse [", line, "]"return CSessionInfoList(list)
Now let's make a letter and send it to ourselves.
defSendMailsessions():print"---"print"Sending statistic to administrator:"
sessions = veeamlpb.session.CSessionInfoList.Get()
recipient = "dear.admin@company.com"
subject = "VAL status notification"
text = "Statistic:\n"
inx = 0;
successCount = 0
warningCount = 0
errorCount = 0for sessionInfo in sessions.List():
if (sessionInfo.State() == "Success"):
successCount += 1elif (sessionInfo.State() == "Warning"):
warningCount += 1else:
errorCount += 1
text += str(successCount)+"/"+str(warningCount)+"/"+str(errorCount)+" Success/Warning/Error\n"
text += "Last 10 session:\n"for sessionInfo in reversed(sessions.List()):
if inx == 10:
text += "...\n"break;
text += str(inx)+" | "+sessionInfo.State()+" | "+sessionInfo.JobName()+" | "+sessionInfo.StartTime()+" / "+sessionInfo.FinishTime() + "\n"#text +=
inx += 1
text += "\n"
text += "--------------------------------------------------------------------------------\n"
text += " Yours sincerely, Veeam Agent for Linux Monitor\n"print text
os.system("echo '"+text+"' | mail -s '"+subject+"' "+recipient)
As a result, after installing mailutils, we can receive a letter of the form:
Statistic:
3/0/0 Success/Warning/Error
Last 10 session:
0 | Success | bj-home | 2018-12-06 06:00 / 2018-12-06 06:00
1 | Success | bj-home | 2018-12-05 16:26 / 2018-12-05 16:26
2 | Success | bj-home | 2018-12-05 15:43 / 2018-12-05 15:44
--------------------------------------------------------------------------------
Yours sincerely, Veeam Agent for Linux Monitor
The letter displays only the last 10 sessions. In this case, at the very beginning of the letter, information about the number of successful and not-so sessions is displayed. Just look at the tsiferki in the letter at the beginning of the working day, checking mail and sipping coffee to understand that the night backups were successful.
If you need something clearer - you can request information about the sessions in xml-format and transfer it to your server. They combine the data into a single summary table that displays all the necessary information in a convenient or convenient format for you.
XML get a couple of lines:
sessionList = veeamlpb.session.CSessionList()
text = sessionList.ToXml()
Save the resulting file
sessionListFileName = "session_list.xml"print"Store XML to file: ",sessionListFileName
sessionListXmlFile = open(sessionListFileName, "w")
sessionListXmlFile.write(text)
sessionListXmlFile.close()
Next, send the resulting XML to the server. An alternative option is also possible - the server collects XML from the machines that are backed up. Who is the initiator is not important for us. It is important that XML servers with session lists from all machines are collected on the server. I chose the first option:
hostname = os.uname()[1]
target = "user@admin-desktop:/home/user"
os.system("scp ./"+sessionListFileName+" "+target+"/backups/"+hostname+"/session_list.xml")
Now on the server side it remains to process the received data and make a beautiful html-page.
import veeamlpb
import os
import datetime
import xml.etree.ElementTree as xml
defmain():
hosts = []
backupsDirectory = "/home/user/backups"for item in os.listdir(backupsDirectory):
if item in [".", ".."]:
continueif os.path.isdir(os.path.join(backupsDirectory,item)):
hosts.append(item)
print"item: ",item
if len(hosts) == 0:
return0
backupSessionMap = {}
for host in hosts:
print"found host: ", host
sessionInfoFile = os.path.join(os.path.join(backupsDirectory,host), "session_list.xml")
sessionList = veeamlpb.session.CSessionInfoList.FromXmlFile(sessionInfoFile)
backupSessionMap[host] = sessionList
for sessionInfo in sessionList.List():
print"Session:",sessionInfo.ToString()
html = xml.Element("html")
body = xml.SubElement(html, "body", {"style":"background-color: #00b336;"})
xml.SubElement(body,"h1").text = "Report at "+datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
xml.SubElement(body,"h2").text = "Statistic:"for host in hosts:
sessionList = backupSessionMap[host]
success=0
warning=0
error=0if len(sessionList.List()) == 0:
continuefor sessionInfo in sessionList.List():
if sessionInfo.State() == "Success":
success +=1elif sessionInfo.State() == "Warning":
warning +=1else:
error +=1
latestSessionInfo = sessionList.List()[-1]
attr = {}
if latestSessionInfo.State() == "Success":
#attr["style"] = "background-color: #00b336;"
attr["style"] = "background-color: #005f4b; color: white;"elif latestSessionInfo.State() == "Warning":
attr["style"] = "background-color: #93ea20;"else:
attr["style"] = "background-color: #ba0200; color: white;"
xml.SubElement(xml.SubElement(body,"p"),"span", attr).text = \
host + " - "+str(success)+"/"+str(warning)+"/"+str(error)+" Success/Warning/Error"for host in hosts:
sessionList = backupSessionMap[host]
xml.SubElement(body,"h2").text = host+":"
tableStyle =xml.SubElement(body,"style")
tableStyle.attrib["type"] = "text/css"
tableStyle.text = "TABLE {border: 1px solid green;} TD{ border: 1px solid green; padding: 4px;}"
table = xml.SubElement(body,"table")
thead = xml.SubElement(table, "thead")
xml.SubElement(thead, "th").text = "Number"
xml.SubElement(thead, "th").text = "State"
xml.SubElement(thead, "th").text = "Job name"
xml.SubElement(thead, "th").text = "Start at"
xml.SubElement(thead, "th").text = "Complete at"
tbody = xml.SubElement(table, "tbody")
inx = 0for sessionInfo in reversed(sessionList.List()):
if inx == 10:
break;
tr = xml.SubElement(tbody,"tr")
xml.SubElement(tr, "td").text = str(inx)
attr ={}
if sessionInfo.State() == "Success":
passelif sessionInfo.State() == "Warning":
attr["style"] ="background-color: #93ea20;"else:
attr["style"] ="background-color: #ba0200; color: white;"
xml.SubElement(tr, "td", attr).text = sessionInfo.State()
xml.SubElement(tr, "td").text = sessionInfo.JobName()
xml.SubElement(tr, "td").text = sessionInfo.StartTime()
xml.SubElement(tr, "td").text = sessionInfo.FinishTime()
inx += 1
xml.ElementTree(html).write("summary.html", encoding='utf-8', method='html')
return0
exit(main())
As a result, the report is ready:
I did not have the task to make a beautiful product. The task was to show that the issue of collecting statistics can be solved on scripts in one or two days.
In principle, if you develop the ideas outlined here, then you can do "Open Backup Monitor for Veeam Agent for Linux". In my opinion, a good topic for coursework in Python, or, maybe, even for a diploma, or just a reason to practice programming in an opensource project. Agree, it is better to practice programming, than to become an 80th-level elf.
All code can be found at http://www.github.com/CodeImp/veeampy/ . Download, use, supplement and fork for health.
Note, the code is distributed under the GPL-2 license, it may contain errors and so on. Everything is as usual in the world of opensource. So before you apply in production - do not forget to drive on the test lab.