August 10, 2010 at 15:25Easy backup automation for Windows
The backup issue is of concern to every specialist responsible for the safety of the invaluable service documents / databases / other files entrusted to him. In order to feel more confident in the face of various likely force majeure, it is necessary to periodically back up critical data. No less urgent is the problem of preserving acquired by overwork and for home users.
We offer a simple but effective script to help solve this problem.
We have the initial conditions:
Operating system: MS Windows XP, 2003, etc .;
Backup objects: Folders and files located on local and network drives in the amount of ~ 200 GB;
Budget: 0 rub .;
Number of specially-trained employees for setting up the process: 1 pers.
Number of specially-trained employees to service the process: 0 people
Objective: daily full backup of data, storage of a given number of created backups without operator intervention.
On the Internet, you can find a fairly large number of different programs that solve a similar problem. According to the task and the set conditions, the following programs were found, installed and tested under conditions close to combat: Cobian Backup (9, 10), Toucan, Areca, FBackup, Backup2Net and the Samsung Auto Backup backup program (comes with the same removable drive) .
Testing the data of programs on working volumes of data showed their steady inability to cope with the simple task of automatically creating daily backups of data of the required volume. Problems arose both at the stage of preparation for the reservation and in the progress thereof.
Cobian Backup 10 lasted the longest (about a week), after which it went into the abandoned one and stubbornly began to backup only about 600 MB of data and fell off with a bunch of errors in the logs. Unfortunately, the manipulations with the settings for shadow copying and archiving did not lead to anything, and as a whole the program left a pretty good impression, so it is recommended for use (provided that it works stable under your specific conditions).
Toucan was particularly hopeful, since the program is related to open source software. But alas, the hopes did not materialize. In the course of the task at different stages, the program fell out with a classic error about an unacceptable operation and the need to close it (program).
The rest of the selection participants also did not suit me for various reasons. I don’t see much point in revealing the testing process of these programs in more detail, because this is not directly related to the purpose of writing this article.
Thus, the simple idea of finding a free solution to the problem was failed miserably. Since there was no strength, no desire, no hope, and backups were absolutely necessary to wander further around the Web for free commercial applications, backups made a natural decision / desire to build something of their own. The result of deliberation over a rather short period of time was the solution proposed below for simple automation of data backup using an ordinary bat script and a standard scheduler. The script works, as they say, “out of the box”, and does not require additional testing and debugging, you only need to provide it with input data and create directories participating in the process in advance (there are no checks on this subject for reasons of simplicity).
Anticipating the dissatisfaction of many about laying out ready-to-use solutions, I want to say that not many of our fellow citizens have the desire / time / opportunity / talent / to enter what they need in order to understand the syntax of the standard Windows command line. I also dare to point out the non-zero probability that at least a few young and green administrators will turn their eyes to the undoubteduselessness of studying this area.
And so, let's get started. First, I will give the full text of the script without comment. For a full discussion with comments, see below. The result of the daily work of the script will be a set of backups of the data you need, which will look something like this: The promised analysis with comments.
In the usual case, to determine the value of the environment variable, you must use the construction% variable%. However, if the value of the variable changes inside the loop and is read there, then for the bat file to work correctly, you should use the SetLocal EnableDelayedExpansion command and use the "!" Symbol instead of the "%" symbol. Otherwise, the value of the variable in the loop will always be the same - the same as it was before entering the loop.
Further, for the proposed script to work successfully, the following input parameters (variables) must be set: The next line starts archiving using the 7-Zip files and folders listed in the list_add.txt file , except for the files specified in the list_excl.txt exclusion file . We will analyze this line in more detail:
C: \ "Program Files" \ 7-Zip \ 7z.exe :: Specifies the location of the executable file of the 7-Zip archiver (more precisely, its console version);
a :: this is the command to add files / folders to the archive;
-t7z :: the key sets the type of the archive being created = 7z;
-mhe = on :: enables password encryption of the archive header and file / folder names in the archive;
The team dynamically generates the full path and file name of the archive of the form “D: \ backups \ files2010-07-12_22-00-00.7z” , where the name contains the date and time of the backup;
@% $ SRC% :: sets the name of the list of archived data; instead of% $ SRC%, the script substitutes the name of the file;
-scsWIN :: sets win encoding for the list file;
-p% $ PSW% :: the key sets the password for encrypting the archive, instead of% $ PSW% the script substitutes the text string of the password;
-mx =% $ COMPR% :: the key sets the compression ratio of the archived data;
-ssw :: key allows archiving files opened by applications for writing;
-xr @% $ EXC% :: sets the name of the file-list of data to be excluded from the archiving process, r = means that subfolders should be excluded, instead of % $ EXC%, the script substitutes the file name;
>> log_7z.log :: characters >> indicate the redirection of messages displayed as a whole line from the screen (standard stream) to the log_7z.lo g file , this is convenient for subsequent analysis of the script;
This part of the script is responsible for the automatic deletion of already unnecessary data archives, in accordance with the amount specified by the $ COPIES variable . Let's analyze in more detail. Automatic deletion is implemented by organizing a cycle during which a set of names (multiple) of archive files is sorted (how it is formed - see the DIR line just below ). In general, our cycle looks like: for / F ["Keyword processing"] { %% | %} variable in (Set of FileNames) do command [CommandString Parameters] The / F switch determines that the loop will parse all elements of the Set of FileName Set ; "Key Words of Parsing"
- indicate which separators are used between the elements of the set being analyzed, in our case there is no separator;
%% i is the variable i , which contains in each iteration step the current element of the set being analyzed (i.e., the next file name in our case).
The command forms a set (set) of file names contained in the folder of the given variable $ DST and corresponding to the mask files ???? - ?? - ?? _ ?? - ?? - ??. 7z , where the question mark denotes any character in this position the / B switch specifies the output of only file names, and the / O: -N switch specifies sorting by name ( N ) or in reverse order ( -N ), in our case, the reverse order.
The set looks like:
files2010-07-01_22-00-00.7z
files2010-06-30_22-00-00.7z
files2010-06-29_22-00-00.7z
Pay attention to the sorting order of elements!
Do (...) - in brackets contains the command (s) that are executed in each iteration, in our case:
Set / A $ COPIES- = 1 :: decrement of the variable COPIES that acts as a loop counter;
If! $ COPIES! LSS 0 DEL "% $ DST% \ %% i" :: the command checks the condition $ COPIES LSS 0 i.e. $ COPIES <0 , if true, then the command to delete the archive file with the name of the current element should be executed, so the number of stored copies of the archive will not exceed the specified value, in our example $ COPIES = 7.
I will also give, for example, the contents of the list_add.txt file : And the list_excl.txt exception file : Actually, this is the whole layout. The script is small but reliable. There are no tricks, everything is quite simple. And finally, banality ... To automate the launch, you can use the standard Windows scheduler, using the command line of the form "d: \ backup \ backup.bat >> d: \ backup \ backup.log". This will allow the script messages to be output to the log file. Good luck to all. References: 1) MS Command Line Reference. technet.microsoft.com/en-us/library/cc737438%28WS.10%29.aspx 2) Detailed help on the functions of the console version of 7-Zip is attached to the archiver.
We offer a simple but effective script to help solve this problem.
We have the initial conditions:
Operating system: MS Windows XP, 2003, etc .;
Backup objects: Folders and files located on local and network drives in the amount of ~ 200 GB;
Budget: 0 rub .;
Number of specially-trained employees for setting up the process: 1 pers.
Number of specially-trained employees to service the process: 0 people
Objective: daily full backup of data, storage of a given number of created backups without operator intervention.
On the Internet, you can find a fairly large number of different programs that solve a similar problem. According to the task and the set conditions, the following programs were found, installed and tested under conditions close to combat: Cobian Backup (9, 10), Toucan, Areca, FBackup, Backup2Net and the Samsung Auto Backup backup program (comes with the same removable drive) .
Testing the data of programs on working volumes of data showed their steady inability to cope with the simple task of automatically creating daily backups of data of the required volume. Problems arose both at the stage of preparation for the reservation and in the progress thereof.
Cobian Backup 10 lasted the longest (about a week), after which it went into the abandoned one and stubbornly began to backup only about 600 MB of data and fell off with a bunch of errors in the logs. Unfortunately, the manipulations with the settings for shadow copying and archiving did not lead to anything, and as a whole the program left a pretty good impression, so it is recommended for use (provided that it works stable under your specific conditions).
Toucan was particularly hopeful, since the program is related to open source software. But alas, the hopes did not materialize. In the course of the task at different stages, the program fell out with a classic error about an unacceptable operation and the need to close it (program).
The rest of the selection participants also did not suit me for various reasons. I don’t see much point in revealing the testing process of these programs in more detail, because this is not directly related to the purpose of writing this article.
Thus, the simple idea of finding a free solution to the problem was failed miserably. Since there was no strength, no desire, no hope, and backups were absolutely necessary to wander further around the Web for free commercial applications, backups made a natural decision / desire to build something of their own. The result of deliberation over a rather short period of time was the solution proposed below for simple automation of data backup using an ordinary bat script and a standard scheduler. The script works, as they say, “out of the box”, and does not require additional testing and debugging, you only need to provide it with input data and create directories participating in the process in advance (there are no checks on this subject for reasons of simplicity).
Anticipating the dissatisfaction of many about laying out ready-to-use solutions, I want to say that not many of our fellow citizens have the desire / time / opportunity / talent / to enter what they need in order to understand the syntax of the standard Windows command line. I also dare to point out the non-zero probability that at least a few young and green administrators will turn their eyes to the undoubted
And so, let's get started. First, I will give the full text of the script without comment. For a full discussion with comments, see below. The result of the daily work of the script will be a set of backups of the data you need, which will look something like this: The promised analysis with comments.
@Echo Off
echo %date% %time% ***Start***
SetLocal EnableDelayedExpansion
Set $SRC=list_add.txt
Set $EXC=list_excl.txt
Set $DST=D:\backups
Set $PSW=password
Set $COMPR=1
Set $COPIES=7
C:\"Program Files"\7-Zip\7z.exe a -t7z -mhe=on "%$DST%\files%DATE:~-4%-%DATE:~3,2%-%DATE:~0,2%_%TIME:~0,2%-%TIME:~3,2%-%TIME:~6,2%.7z" @%$SRC% -scsWIN -p%$PSW% -mx=%$COMPR% -ssw -xr@%$EXC% >> log_7z.log
For /F "Delims=" %%i In ('DIR /B/O:-N %$DST%\files????-??-??_??-??-??.7z') Do (
Set /A $COPIES-=1
If !$COPIES! LSS 0 DEL "%$DST%\%%i"
)
echo %date% %time% ***End***dir d:\backups\
files2010-07-12_22-00-00.7z
files2010-07-13_22-00-00.7z
files2010-07-14_22-00-00.7z
files2010-07-15_22-00-00.7z
files2010-07-16_22-00-00.7z@Echo Off :: Отключение вывода текста скрипта
echo %date% %time% ***Start*** ::Вывод даты и времени старта скрипта
SetLocal EnableDelayedExpansion :: Команда изменяет работу с переменными в цикле. In the usual case, to determine the value of the environment variable, you must use the construction% variable%. However, if the value of the variable changes inside the loop and is read there, then for the bat file to work correctly, you should use the SetLocal EnableDelayedExpansion command and use the "!" Symbol instead of the "%" symbol. Otherwise, the value of the variable in the loop will always be the same - the same as it was before entering the loop.
Further, for the proposed script to work successfully, the following input parameters (variables) must be set: The next line starts archiving using the 7-Zip files and folders listed in the list_add.txt file , except for the files specified in the list_excl.txt exclusion file . We will analyze this line in more detail:
$SRC=list_add.txt ::файл-список архивируемых данных
$EXC=list_excl.txt ::файл-список исключаемых файлов
$DST=D:\backups ::место хранения резервных копий
$PSW=password ::секретный пароль шифрования архива
$COMPR=1 ::степень сжатия данных архиватором 7zip (1=самое быстрое архивирование, 5=нормальное сжатие, 9=ультра-сжатие, 0=архивация без сжатия)
$COPIES=7 ::количество хранимых резервных копийC:\"Program Files"\7-Zip\7z.exe a -t7z -mhe=on "%$DST%\files%DATE:~-4%-%DATE:~3,2%-%DATE:~0,2%_%TIME:~0,2%-%TIME:~3,2%-%TIME:~6,2%.7z" @%$SRC% -scsWIN -p%$PSW% -mx=%$COMPR% -ssw -xr@%$EXC% >> log_7z.logC: \ "Program Files" \ 7-Zip \ 7z.exe :: Specifies the location of the executable file of the 7-Zip archiver (more precisely, its console version);
a :: this is the command to add files / folders to the archive;
-t7z :: the key sets the type of the archive being created = 7z;
-mhe = on :: enables password encryption of the archive header and file / folder names in the archive;
%$DST%\files%DATE:~-4%-%DATE:~3,2%-%DATE:~0,2%_%TIME:~0,2%-%TIME:~3,2%-%TIME:~6,2%.7z The team dynamically generates the full path and file name of the archive of the form “D: \ backups \ files2010-07-12_22-00-00.7z” , where the name contains the date and time of the backup;
@% $ SRC% :: sets the name of the list of archived data; instead of% $ SRC%, the script substitutes the name of the file;
-scsWIN :: sets win encoding for the list file;
-p% $ PSW% :: the key sets the password for encrypting the archive, instead of% $ PSW% the script substitutes the text string of the password;
-mx =% $ COMPR% :: the key sets the compression ratio of the archived data;
-ssw :: key allows archiving files opened by applications for writing;
-xr @% $ EXC% :: sets the name of the file-list of data to be excluded from the archiving process, r = means that subfolders should be excluded, instead of % $ EXC%, the script substitutes the file name;
>> log_7z.log :: characters >> indicate the redirection of messages displayed as a whole line from the screen (standard stream) to the log_7z.lo g file , this is convenient for subsequent analysis of the script;
This part of the script is responsible for the automatic deletion of already unnecessary data archives, in accordance with the amount specified by the $ COPIES variable . Let's analyze in more detail. Automatic deletion is implemented by organizing a cycle during which a set of names (multiple) of archive files is sorted (how it is formed - see the DIR line just below ). In general, our cycle looks like: for / F ["Keyword processing"] { %% | %} variable in (Set of FileNames) do command [CommandString Parameters] The / F switch determines that the loop will parse all elements of the Set of FileName Set ; "Key Words of Parsing"
For /F "Delims=" %%i In ('DIR /B/O:-N %$DST%\files????-??-??_??-??-??.7z') Do (
Set /A $COPIES-=1
If !$COPIES! LSS 0 DEL "%$DST%\%%i"
)- indicate which separators are used between the elements of the set being analyzed, in our case there is no separator;
%% i is the variable i , which contains in each iteration step the current element of the set being analyzed (i.e., the next file name in our case).
DIR /B/O:-N %$DST%\files????-??-??_??-??-??.7zThe command forms a set (set) of file names contained in the folder of the given variable $ DST and corresponding to the mask files ???? - ?? - ?? _ ?? - ?? - ??. 7z , where the question mark denotes any character in this position the / B switch specifies the output of only file names, and the / O: -N switch specifies sorting by name ( N ) or in reverse order ( -N ), in our case, the reverse order.
The set looks like:
files2010-07-01_22-00-00.7z
files2010-06-30_22-00-00.7z
files2010-06-29_22-00-00.7z
Pay attention to the sorting order of elements!
Do (...) - in brackets contains the command (s) that are executed in each iteration, in our case:
Set / A $ COPIES- = 1 :: decrement of the variable COPIES that acts as a loop counter;
If! $ COPIES! LSS 0 DEL "% $ DST% \ %% i" :: the command checks the condition $ COPIES LSS 0 i.e. $ COPIES <0 , if true, then the command to delete the archive file with the name of the current element should be executed, so the number of stored copies of the archive will not exceed the specified value, in our example $ COPIES = 7.
echo %date% %time% ***End*** ::Вывод даты и времени окончания работы скриптаI will also give, for example, the contents of the list_add.txt file : And the list_excl.txt exception file : Actually, this is the whole layout. The script is small but reliable. There are no tricks, everything is quite simple. And finally, banality ... To automate the launch, you can use the standard Windows scheduler, using the command line of the form "d: \ backup \ backup.bat >> d: \ backup \ backup.log". This will allow the script messages to be output to the log file. Good luck to all. References: 1) MS Command Line Reference. technet.microsoft.com/en-us/library/cc737438%28WS.10%29.aspx 2) Detailed help on the functions of the console version of 7-Zip is attached to the archiver.
M:\mail
U:\updates
Z:\Бухгалтерия
Z:\Отдел*
z:\Справка
Z:\Сисадмин
*.bak
*.tmp
*.avi
*.mkv
*.mp3