Fast and reliable data backup to the cloud 2
- Tutorial
In my previous article, I described an example of a script that can archive and upload a certain amount of user files to the cloud without its participation, thereby ensuring the simplest backup of user files. The script closes the following questions:
Naturally, the simplicity of the script turned its flaws, namely:
In this regard, the script code of the full and incremental archive has been added with the necessary functionality. For a general description, see the previous article , here I want to dwell only on the changes.
call pass.bat xx x - calls an external program that returns the variable xx back with the value specified for the identifier x.
Accordingly, the user variable is assigned the value associated with the identifier 1 (username of the account yandex), pass - 2 (password of the account yandex), pwd - 3 (password to the archive), recipient - 4 (mail, where to send the operation log).
The pass.bat script is given conditionally, the password protection method can be chosen differently, but for those and this option is appropriate, I quote the simplest code of this script:
When calling this code, it finds next to itself the file pass.txt, and assigns the variable whose name is specified in the first argument of the script call to the value of that line of the pass.txt file whose number is specified in the second argument of the script call. The location of the pass.bat and pass.txt files is not critical, as long as they are in the same directory as specified in the system path variable.
set mailsend = mailsend1.17b15.exe - an indication of the program for sending a report to the user's mail. The letter is sent from the account yandex.ru, from which the files are sent to the cloud.
set i = 1 - setting the retry count to 1
call: loging "-----------------------------------"- call the logging procedure. Procedure requires a string, which is then written to the file 2 - log file % filebkp% -log.txt and separate file current account % filebkp -log.tmp% , which is then sent to the user as a result of the archiving report.
Accordingly, logging lines in the body of scripts simultaneously show the user the operations performed during operation, write them to the logs and for me are comments in the body of the script, explaining what is being done on the following lines.
Accordingly, a full copy script and an incremental copy script perform the following operations:
These scripts are given only as examples, there is no desire to rewrite under PowerShell, since It is important for me that it works on any system, including Windows XP. There is no protection against password theft; I’d be very interested in what security experts would suggest, but separating the code and passwords with the possibility of placing the passwords in another directory at least removes the possibility of a stupid mistake that when sending a script to a friend, he will also get passwords.
- fast (perfect in the background);
- would save both the current version of the files and the previous ones;
- file storage would not be available to viruses or natural disasters such as killing a carrier or stealing it;
- the contents of the backup would be available only to the user, i.e. has been polished.
Naturally, the simplicity of the script turned its flaws, namely:
- Storing passwords in the body of the script. Yes, many programs store passwords unencrypted in text files, and protection of this data occurs through file access control, but still removing the password from the code is a sign of good tone and minimal protection in case the working script somehow falls into the wrong hands .
- Lack of verification of successful completion of the operation, and rollback to the previous state in case the operation was not successfully completed.
- The lack of logging and notifying the user about the success of the operation, if the script is launched through the Task Scheduler.
In this regard, the script code of the full and incremental archive has been added with the necessary functionality. For a general description, see the previous article , here I want to dwell only on the changes.
The script to create a complete archive
@echo off
chcp1251 >nulset filebkp=server
call pass.bat user 1call pass.bat pass 2call pass.bat pwd 3call pass.bat recipient 4set mailsend=mailsend1.17b15.exe
set i=1set pathbkp=..\backup
set pathtemp=..\temp
set srvbkp=https://%user%:%pass%@webdav.yandex.ru/backup/%filebkp%set full=%filebkp%-full
set inc=%filebkp%-inc
set h=%TIME:~0,2%set m=%TIME:~3,2%set s=%TIME:~6,2%set ms=%TIME:~9,2%set curtime=%h%-%m%-%s%set dd=%DATE:~0,2%set mm=%DATE:~3,2%set yyyy=%DATE:~6,4%set curdate=%yyyy%-%mm%-%dd%set curdatetime=%curdate: =0%-%curtime: =0%
call :loging "-----------------------------------"
call :loging "Текущее время - %curdatetime%"
call :loging "Создаем резервную копию текущих архивов"
ren "%pathtemp%" temp-bkp
md "%pathtemp%"
call :loging "Создаем полный архив %full%.7z"
7z.exe a -xr0!*.log -xr0!*.bak -xr0!*.tmp "%pathtemp%\%full%".7z -r -mx1 "%pathbkp%\*" -%pwd%call :loging "Создаем резервную копию текущих архивов на сервере"
curl.exe -k -X MOVE -H "Destination:/backup/%filebkp%"-bkp "%srvbkp%" -o .\stdout
curl.exe -k -X MKCOL "%srvbkp%" -o .\stdout
:upload
call :loging "Попытка отправки номер %i%"
call :loging "Отправляем на сервер файл %full%.7z"
curl.exe -k -T "%pathtemp%\%full%".7z "%srvbkp%"/ --progress-bar -o .\stdout
call :loging "Скачиваем файл %full%.7z с сервера для сравнения с исходным"
curl.exe -k "%srvbkp%/%full%".7z -o "%pathtemp%"\test.7z --progress-bar -o .\stdout
call :loging "Сравниваем скачанный файл с исходным"
fc /LB1 /B "%pathtemp%\%full%".7z "%pathtemp%"\test.7z > nulifERRORLEVEL1goto Different
call :loging "Файлы одинаковы:"
set tolog="%full%.7z успешно отправлен на сервер"
call :loging %tolog%set mailsubject=%tolog:"=%call :loging "Удаляем устаревшие версии архива с компьютера и сервера"
RD /s/q "%pathtemp%"-bkp
curl.exe -k -X DELETE "%srvbkp%"-bkp -o .\stdout
del /q "%pathtemp%"\test.7z
gotoexit
:Different
call :loging "Файлы различаются: %full%.7z на сервере НЕ совпадает с исходным"
call :loging "Повторяем отправку"
set /a i+=1if%i% = 5gotoexit-f
goto upload
:exit-f
set tolog="Отправка файла %full%.7z на сервер не удалась"
call :loging %tolog%set mailsubject=%tolog:"=%call :loging "Восстанавливаем файлы предыдущего архива"
RD /s/q "%pathtemp%"
ren "%pathtemp%"-bkp temp
curl.exe -k -X DELETE "%srvbkp%" -o .\stdout
curl.exe -k -X MOVE -H "Destination:/backup/%filebkp%" "%srvbkp%"-bkp -o .\stdout
:exitecho Отправляем лог %filebkp%-log.txt на сервер
curl.exe -k -T %filebkp%-log.txt "%srvbkp%"/../ --progress-bar -o .\stdout
echo Отправляем отчет в почту
%mailsend% -to %recipient% -from %user%@yandex.ru -ssl -port 465 -auth -smtp smtp.yandex.ru -user %user% -pass %pass% -sub "%filebkp%: %curdatetime% - %mailsubject%" -cs "windows-1251" -mime-type "text/plain" -msg-body %filebkp%-log.tmp
del /q %filebkp%-log.tmp
exit
:loging
echo %~1echo %~1 >> %filebkp%-log.txt
echo %~1 >> %filebkp%-log.tmp
exit /b
Script of creation of incremental archive
@echo off
chcp1251 >nulset filebkp=server
call pass.bat user 1call pass.bat pass 2call pass.bat pwd 3call pass.bat recipient 4set mailsend=mailsend1.17b15.exe
set tolog=zero
set mailsubject=zero
set i=1set pathbkp=..\backup
set pathtemp=..\temp
set srvbkp=https://%user%:%pass%@webdav.yandex.ru/backup/%filebkp%set full=%filebkp%-full
set inc=%filebkp%-inc
set h=%TIME:~0,2%set m=%TIME:~3,2%set s=%TIME:~6,2%set ms=%TIME:~9,2%set curtime=%h%-%m%-%s%set dd=%DATE:~0,2%set mm=%DATE:~3,2%set yyyy=%DATE:~6,4%set curdate=%yyyy%-%mm%-%dd%set curdatetime=%curdate: =0%-%curtime: =0%
call :loging "-----------------------------------"
call :loging "Текущее время - %curdatetime%"
call :loging "Создаем инкрементный архив %inc%-%curdatetime%.7z"
7z.exe u -xr0!*.log -xr0!*.bak -xr0!*.tmp "%pathtemp%\%full%".7z -u- -up3q3r2x2y2z0w2!"%pathtemp%\%inc%-%curdatetime%".7z "%pathbkp%\*" -%pwd%
:upload
call :loging "Попытка отправки номер %i%"
call :loging "Отправляем архив на сервер"
curl.exe -k -T "%pathtemp%\%inc%-%curdatetime%".7z "%srvbkp%"/ --progress-bar -o .\stdout
call :loging "Скачиваем архив с сервера для сравнения с исходным"
curl.exe -k "%srvbkp%/%inc%-%curdatetime%".7z -o "%pathtemp%"\test.7z --progress-bar -o .\stdout
call :loging "Сравниваем скачанный архив с исходным"
fc /LB1 /B "%pathtemp%\%inc%-%curdatetime%".7z "%pathtemp%"\test.7z > nulifERRORLEVEL1goto Different
call :loging "Файлы одинаковы"
set tolog="%inc%-%curdatetime%.7z успешно отправлен на сервер"
call :loging %tolog%set mailsubject=%tolog:"=%gotoexit
:Different
call :loging "Файлы различаются: %inc%-%curdatetime%.7z на сервере НЕ совпадает с исходным"
call :loging "Повторяем отправку"
set /a i+=1if%i% = 5gotoexit-f
goto upload
:exit-f
set tolog="Отправка файла %inc%-%curdatetime%.7z на сервер не удалась"
call :loging %tolog%set mailsubject=%tolog:"=%
:exitdel /q "%pathtemp%"\test.7z
echo Отправляем лог %filebkp%-log.txt на сервер
curl.exe -k -T %filebkp%-log.txt "%srvbkp%"/../ --progress-bar -o .\stdout
echo Отправляем отчет в почту
%mailsend% -to %recipient% -from %user%@yandex.ru -ssl -port 465 -auth -smtp smtp.yandex.ru -user %user% -pass %pass% -sub "%filebkp%: %mailsubject%" -cs "windows-1251" -mime-type "text/plain" -msg-body %filebkp%-log.tmp
del /q %filebkp%-log.tmp
exit
:loging
echo %~1echo %~1 >> %filebkp%-log.txt
echo %~1 >> %filebkp%-log.tmp
exit /b
New teams:
call pass.bat xx x - calls an external program that returns the variable xx back with the value specified for the identifier x.
Accordingly, the user variable is assigned the value associated with the identifier 1 (username of the account yandex), pass - 2 (password of the account yandex), pwd - 3 (password to the archive), recipient - 4 (mail, where to send the operation log).
The pass.bat script is given conditionally, the password protection method can be chosen differently, but for those and this option is appropriate, I quote the simplest code of this script:
for /f "usebackq delims=" %%iin (`find /n /v "" "%~dp0"pass.txt ^| find "[%2]"`) do (
set newvar=%%i
)
set%1=%newvar:~3%
When calling this code, it finds next to itself the file pass.txt, and assigns the variable whose name is specified in the first argument of the script call to the value of that line of the pass.txt file whose number is specified in the second argument of the script call. The location of the pass.bat and pass.txt files is not critical, as long as they are in the same directory as specified in the system path variable.
set mailsend = mailsend1.17b15.exe - an indication of the program for sending a report to the user's mail. The letter is sent from the account yandex.ru, from which the files are sent to the cloud.
set i = 1 - setting the retry count to 1
call: loging "-----------------------------------"- call the logging procedure. Procedure requires a string, which is then written to the file 2 - log file % filebkp% -log.txt and separate file current account % filebkp -log.tmp% , which is then sent to the user as a result of the archiving report.
Accordingly, logging lines in the body of scripts simultaneously show the user the operations performed during operation, write them to the logs and for me are comments in the body of the script, explaining what is being done on the following lines.
Accordingly, a full copy script and an incremental copy script perform the following operations:
- for full archive - creates backup copies of current archives on the server and on the local computer
- creates a new backup
- trying to send it to the server
- after sending downloads a backup and compares with the original
- in case of success of the comparison — sends a log of operations to the server and a user in the mail a report on a successful operation through the program mailsend1.17b15.exe + for a full archive — deletes outdated versions of the archive
- in case of detection of differences between files - tries to resend the file to the server, and if it fails to send after 5 attempts - sends a log of operations to the server and the user to the mail via the mailsend1.17b15.exe program:
- cleans up a temporary file of current operations.
PS
These scripts are given only as examples, there is no desire to rewrite under PowerShell, since It is important for me that it works on any system, including Windows XP. There is no protection against password theft; I’d be very interested in what security experts would suggest, but separating the code and passwords with the possibility of placing the passwords in another directory at least removes the possibility of a stupid mistake that when sending a script to a friend, he will also get passwords.