Databene Benerator - energize it
“Take out the dictionary and see what catharsis is.” If this is what he wants to kill us with, I want to know what it is. ” (c) Analyze it!
Late in the evening, when the design of the database for 64 tables was almost completed, and the interface for filling them had not even begun, the question arose of how to fill them with data.
Fill in manually - the idea was thrown aside right away.
“Something must be done!” - screamed the soul.
“You need to download something!” - insisted the mind!
As a result, I scoured the Internet and found about a dozen different kinds of solutions, both installed and SaaS, both paid and free - I found it - a databene-benerator- generator of related data (fixtures) for databases. And an article in Russian, with a description of the features and syntax (1), as well as her, but in English (2). I understand - this is what you need! But where to get it? How to use it under Windows? Conveniently? With support for Russian characters?
And so “catharsis” (3) is a concept in ancient philosophy; A term for the process and result of the facilitating, cleansing and ennobling effect on a person of various factors.
How does this relate to the topic of publication? You will understand if you read this. I invite you to cut!
What was described in the aforementioned articles on the “generator” did not suit me at all.
If at least one of the above is suitable for you, it means that you and I have a different path, or rather a different entrance to this path!
First, we need to get the generator itself, for this we need to fill out the form at:
bergmann-it.de/download/download_benerator?lang=en
and click "Download".
At the time of publication, 0.9.8 is available, but I used 0.9.7, in fact, you most likely will not notice the difference, since the latest manual that I could find is this one (4) for version 0.8.1.
I stumbled upon it quite by accident, comparing the version in the manual (http://databene.org/download/databene-benerator-manual-0.7.6.pdf) on the site and the version of the generator. I began to select the version in the address of the manual, and what a surprise it was to find 0.8.1 !!! Further searches were unsuccessful ...
And so you did it! In our hands, i.e. at your fingertips archive "databene-benerator-0.9.7" (you have fresh). Now what to do about it.
Unpack it in "D: \ databene-benerator-0.9.7".
And then shamanism went pure: maven is mentioned on the forums - I don’t know who this beast is, but I’ll say that it works without it!
By not tricky operation, I look at what is in the archive. There are batch files (or sh scripts by the way, too) that start something ... benerator.bat starts benerator_common.bat, it starts java.exe. In the parameters of the first benerator.xml. In the second path to the lib folder, and there * .jar. ...
At that time I tried to work only in two IDEs for Java development - Netbeans and Eclipse. Asked Google the question "databene Benerator eclipse" and in the issue received an answer "databene.org/databene-benerator/115-my-first-ide-based-benerator-project.html "- but there are no links from the pages of the official site to this page!
Now we need Eclipse, download and unpack if it is not already. Any version will do. I am a little familiar with PHP, so you will guess my choice. By the way, the location of the Eclipse working windows (the so-called perspective) for working with the generator is more convenient than PHP (you can select it in the upper right corner).
And so we start Eclipse, create a project:
Select “File-> New-> Project ...”
Then select “Java Project”, click “Next->”.
In the window that appears, enter the name of the project “generatedb”, and select Project layout as “Use project folder as root for sources and class files”, click “Next->”.
Switch to the Libraries tab, click "Add External JARs ...". In the window that opens, go to "D: \ databene-benerator-0.9.7 \ lib" and select all the files that are there.
Click Finish. The project is created.
However, to start the generator, you must configure the "starter"!
Select "Run-> Run Configurations ...".
In the window that appears:
1. On “Java Aplication”, make RMB and select “New”.
2. Then “Name” indicate the name of our configuration to run.
3. Next, “Project” is left unchanged.
4. And in the "Main class" we enter "org.databene.benerator.main.Benerator".
5. Click "Apply."
If you still clicked “Run,” then a large number of lines of different curses will be displayed in the Console tab, all of which are not in Russian. This is because we have not done the most important thing. So what are we waiting for?
It is time to add the files “benerator.xml” and “log4j.xml” to our project, the absence of which benerator swore.
RMB on the project in the project explorer and select “New-> XML File”, enter the file name, then “Finish”.
benerator.xml - the main project file, it describes everything that you will do with your tables.
log4j.xml - configuration file of the tool for logs, it depends on its configuration what and how much the generator will spit out to the console (service information).
The contents of "log4j.xml" is reduced to the form:
The contents of "benerator.xml" are reduced to the form:
Let us dwell on some points of “benerator.xml”, it is considered in more detail in the first Russian-language article and in the manual for version 0.8.1.
Here I note that the magic line “characterEncoding = UTF-8” in the url parameter solves the problem of transferring Russian characters to the database (and not only Russian ones).
The developers on the site also forgot to mention this. Well, that’s not their concern. The jdbc driver configuration line is universal for Java applications, as a result, I found it somewhere on a non-search resource.
Clearing the database tables before re-generating
First, prepare the file “truncate_tables.mysql.sql” (text)
The first and last lines are disabling and enabling table consistency checks. Otherwise, a situation may arise when one table blocks the deletion of records by another (referential integrity).
Lines - commands for cleaning specific database tables. It is advisable to group them according to the following principle:
1. A separate group of directories (not connected with any others)
2. Further, groups can be cascaded into already completed tables.
Cleaning - either as a whole group, or one at a time, but monitor dependencies.
Commenting on the table is convenient so that you can flexibly perform the repeated generation of data.
In benerator.xml, after defining memstore, add the lines:
Be carefull! After starting the generator - not commented tables will be cleared!
Deleting tables and creating a database - making such manipulations here, in my opinion, is not worth it. There are convenient tools for synchronizing the model and the database. I use MySQL Workbench (5).
Embedding scripts into ... JavaScript !? Yes, yes on it!
In benerator.xml, after determining execute, add the lines:
Create the file “script.js” (text)
The first function performs transliteration of Russian characters into English (taken from (5), with minor revisions).
The second - cuts a piece of string.
An example of using JavaScript in code:
The main idea: everything in the script parameter inside {js:} is the essence of JavaScript. Variables are transmitted transparently, in other ways this can be seen from examples of use.
Have you already noticed the abbreviated notation of the conditional if statement?
Distribution of database tables into separate files for ease of generation
It was convenient for me to select each table or group of 2-3 interconnected tables, the generation of which cannot be performed independently - into a separate file “* .ben.xml”. Each file is commented separately for the convenience of its separate generation.
Please note: these files must have the extension * .ben.xml.
In the main file, it looks like this:
Example file "table.s_organization.ben.xml" (XML)
Please note that the structure is similar to “benerator.xml”, however, you do not need to describe the connection to the database and the connection of common modules, as all this has already been done in the main configuration file.
Now, why did I experience the “catharsis” - after all this torment, everything worked:
1. The databene-benerator started and filled the tablets with data, only 2-3 nights and voila a convenient tool for solving an urgent task!
2. It turns out that he understands Russian characters, and this is my cant that I have little knowledge of the syntax for accessing the jdbc driver in Java projects (the syntax is universal) - another 3 nights and everything is fine too!
3. Algorithms for filling the tables went one by one, they surrendered under my pressure every evening. And all 64 tables were completed in 6 evenings.
Yes, there are many more questions, but the main ones are revealed, the task is completed, knowledge is gained, experience is gained. To change the number and quality of entries in the tables, I do not need to “shovel” their hands. The generator will do its job in a few minutes.
The article does not consider:
1. generation of interconnected tables,
2. work with date and time,
3. generation of real numbers.
However, this information is in the posts to which there are links, as well as in the documentation. So after such an acceleration, the reader does not have much difficulty to master these issues.
Especially for the article, I registered on github and posted the sourcesthat can help sort out the examples. To use them, just download the * .zip archive, unzip it. Create a new project and import into it “File-> Import-> General-> FileSystem”. Mark the entire project and click Finish. Remember to add a “launcher” and generator libraries.
Thanks for attention!
1. habrahabr.ru/post/169713 . [Online]
2. sysmagazine.com/posts/169713 . [Online]
3. ru.wikipedia.org/wiki/Katarsis . [Online]
4. databene.org/download/databene-benerator-manual-0.8.1.pdf . [Online]
5. dev.mysql.com/downloads/workbench . [On the web]
6. ajaxs.ru/lesson/javascript/137-transliteracija_stroki_na_javascript.html . [In the Internet]
Introduction
Late in the evening, when the design of the database for 64 tables was almost completed, and the interface for filling them had not even begun, the question arose of how to fill them with data.
Fill in manually - the idea was thrown aside right away.
“Something must be done!” - screamed the soul.
“You need to download something!” - insisted the mind!
As a result, I scoured the Internet and found about a dozen different kinds of solutions, both installed and SaaS, both paid and free - I found it - a databene-benerator- generator of related data (fixtures) for databases. And an article in Russian, with a description of the features and syntax (1), as well as her, but in English (2). I understand - this is what you need! But where to get it? How to use it under Windows? Conveniently? With support for Russian characters?
And so “catharsis” (3) is a concept in ancient philosophy; A term for the process and result of the facilitating, cleansing and ennobling effect on a person of various factors.
How does this relate to the topic of publication? You will understand if you read this. I invite you to cut!
Creating a project in Eclipse
What was described in the aforementioned articles on the “generator” did not suit me at all.
- I use Windows to work;
- I love the GUI (such a weakness as cats ... Well, you understand).
- I can work with MySQL, but not with PostgreSQL.
- I also need data in Russian.
If at least one of the above is suitable for you, it means that you and I have a different path, or rather a different entrance to this path!
First, we need to get the generator itself, for this we need to fill out the form at:
bergmann-it.de/download/download_benerator?lang=en
and click "Download".
At the time of publication, 0.9.8 is available, but I used 0.9.7, in fact, you most likely will not notice the difference, since the latest manual that I could find is this one (4) for version 0.8.1.
I stumbled upon it quite by accident, comparing the version in the manual (http://databene.org/download/databene-benerator-manual-0.7.6.pdf) on the site and the version of the generator. I began to select the version in the address of the manual, and what a surprise it was to find 0.8.1 !!! Further searches were unsuccessful ...
And so you did it! In our hands, i.e. at your fingertips archive "databene-benerator-0.9.7" (you have fresh). Now what to do about it.
Unpack it in "D: \ databene-benerator-0.9.7".
And then shamanism went pure: maven is mentioned on the forums - I don’t know who this beast is, but I’ll say that it works without it!
By not tricky operation, I look at what is in the archive. There are batch files (or sh scripts by the way, too) that start something ... benerator.bat starts benerator_common.bat, it starts java.exe. In the parameters of the first benerator.xml. In the second path to the lib folder, and there * .jar. ...
At that time I tried to work only in two IDEs for Java development - Netbeans and Eclipse. Asked Google the question "databene Benerator eclipse" and in the issue received an answer "databene.org/databene-benerator/115-my-first-ide-based-benerator-project.html "- but there are no links from the pages of the official site to this page!
Now we need Eclipse, download and unpack if it is not already. Any version will do. I am a little familiar with PHP, so you will guess my choice. By the way, the location of the Eclipse working windows (the so-called perspective) for working with the generator is more convenient than PHP (you can select it in the upper right corner).
And so we start Eclipse, create a project:
Select “File-> New-> Project ...”
Then select “Java Project”, click “Next->”.
In the window that appears, enter the name of the project “generatedb”, and select Project layout as “Use project folder as root for sources and class files”, click “Next->”.
Switch to the Libraries tab, click "Add External JARs ...". In the window that opens, go to "D: \ databene-benerator-0.9.7 \ lib" and select all the files that are there.
Click Finish. The project is created.
However, to start the generator, you must configure the "starter"!
Select "Run-> Run Configurations ...".
In the window that appears:
1. On “Java Aplication”, make RMB and select “New”.
2. Then “Name” indicate the name of our configuration to run.
3. Next, “Project” is left unchanged.
4. And in the "Main class" we enter "org.databene.benerator.main.Benerator".
5. Click "Apply."
If you still clicked “Run,” then a large number of lines of different curses will be displayed in the Console tab, all of which are not in Russian. This is because we have not done the most important thing. So what are we waiting for?
Project structure
It is time to add the files “benerator.xml” and “log4j.xml” to our project, the absence of which benerator swore.
RMB on the project in the project explorer and select “New-> XML File”, enter the file name, then “Finish”.
benerator.xml - the main project file, it describes everything that you will do with your tables.
log4j.xml - configuration file of the tool for logs, it depends on its configuration what and how much the generator will spit out to the console (service information).
The contents of "log4j.xml" is reduced to the form:
log4j.xml
The contents of "benerator.xml" are reduced to the form:
benerator.xml
The main scenario and some working solutions
Let us dwell on some points of “benerator.xml”, it is considered in more detail in the first Russian-language article and in the manual for version 0.8.1.
Here I note that the magic line “characterEncoding = UTF-8” in the url parameter solves the problem of transferring Russian characters to the database (and not only Russian ones).
The developers on the site also forgot to mention this. Well, that’s not their concern. The jdbc driver configuration line is universal for Java applications, as a result, I found it somewhere on a non-search resource.
Clearing the database tables before re-generating
First, prepare the file “truncate_tables.mysql.sql” (text)
truncate_tables.mysql.sql
SET foreign_key_checks = 0;
--truncate table s_person;
--truncate table s_job_title;
--truncate table s_organization;
--truncate table s_department;
--truncate table t_orgstructure;
--truncate table s_type_project;
--truncate table s_direction_project;
--truncate table s_norm_labor;
--truncate table s_timetable;
SET foreign_key_checks = 1;
The first and last lines are disabling and enabling table consistency checks. Otherwise, a situation may arise when one table blocks the deletion of records by another (referential integrity).
Lines - commands for cleaning specific database tables. It is advisable to group them according to the following principle:
1. A separate group of directories (not connected with any others)
2. Further, groups can be cascaded into already completed tables.
Cleaning - either as a whole group, or one at a time, but monitor dependencies.
Commenting on the table is convenient so that you can flexibly perform the repeated generation of data.
In benerator.xml, after defining memstore, add the lines:
Подготавливаем базу данных Be carefull! After starting the generator - not commented tables will be cleared!
Deleting tables and creating a database - making such manipulations here, in my opinion, is not worth it. There are convenient tools for synchronizing the model and the database. I use MySQL Workbench (5).
Embedding scripts into ... JavaScript !? Yes, yes on it!
In benerator.xml, after determining execute, add the lines:
Подготавливаем внешние скрипты Create the file “script.js” (text)
script.js
function toLink (str) {
var space = '';
str = str.toLowerCase();
var transl = {
'а': 'a', 'б': 'b', 'в': 'v', 'г': 'g', 'д': 'd', 'е': 'e', 'ё': 'e', 'ж': 'zh',
'з': 'z', 'и': 'i', 'й': 'j', 'к': 'k', 'л': 'l', 'м': 'm', 'н': 'n',
'о': 'o', 'п': 'p', 'р': 'r','с': 's', 'т': 't', 'у': 'u', 'ф': 'f', 'х': 'h',
'ц': 'c', 'ч': 'ch', 'ш': 'sh', 'щ': 'sh','ъ': space, 'ы': 'y', 'ь': space, 'э': 'e', 'ю': 'yu', 'я': 'ya'
}
var link = '';
for (var i = 0; i < str.length; i++) {
if(/[а-яё]/.test(str.charAt(i))) { //если текущий символ - русская буква, то меняем его
link += transl[str.charAt(i)];
} else if (/[a-z0-9]/.test(str.charAt(i))) {
link += str.charAt(i); //если текущий символ - английская буква или цифра, то оставляем как есть
} else {
if (link.slice(-1) !== space) link += space; // если не то и не другое то вставляем space
}
}
return link;
}
function cut(str, cutStart, cutEnd){
return str.substr(cutStart,cutEnd);
}
The first function performs transliteration of Russian characters into English (taken from (5), with minor revisions).
The second - cuts a piece of string.
An example of using JavaScript in code:
…
…
The main idea: everything in the script parameter inside {js:} is the essence of JavaScript. Variables are transmitted transparently, in other ways this can be seen from examples of use.
Have you already noticed the abbreviated notation of the conditional if statement?
Distribution of database tables into separate files for ease of generation
It was convenient for me to select each table or group of 2-3 interconnected tables, the generation of which cannot be performed independently - into a separate file “* .ben.xml”. Each file is commented separately for the convenience of its separate generation.
Please note: these files must have the extension * .ben.xml.
In the main file, it looks like this:
-->
-->
-->Example file "table.s_organization.ben.xml" (XML)
table.s_organization.ben.xml
[[POPULATE TABLE s_organization]] Processed... [[POPULATE TABLE s_organization]] End. OK! Please note that the structure is similar to “benerator.xml”, however, you do not need to describe the connection to the database and the connection of common modules, as all this has already been done in the main configuration file.
Conclusion
Now, why did I experience the “catharsis” - after all this torment, everything worked:
1. The databene-benerator started and filled the tablets with data, only 2-3 nights and voila a convenient tool for solving an urgent task!
2. It turns out that he understands Russian characters, and this is my cant that I have little knowledge of the syntax for accessing the jdbc driver in Java projects (the syntax is universal) - another 3 nights and everything is fine too!
3. Algorithms for filling the tables went one by one, they surrendered under my pressure every evening. And all 64 tables were completed in 6 evenings.
Yes, there are many more questions, but the main ones are revealed, the task is completed, knowledge is gained, experience is gained. To change the number and quality of entries in the tables, I do not need to “shovel” their hands. The generator will do its job in a few minutes.
The article does not consider:
1. generation of interconnected tables,
2. work with date and time,
3. generation of real numbers.
However, this information is in the posts to which there are links, as well as in the documentation. So after such an acceleration, the reader does not have much difficulty to master these issues.
Especially for the article, I registered on github and posted the sourcesthat can help sort out the examples. To use them, just download the * .zip archive, unzip it. Create a new project and import into it “File-> Import-> General-> FileSystem”. Mark the entire project and click Finish. Remember to add a “launcher” and generator libraries.
Thanks for attention!
Materials used
1. habrahabr.ru/post/169713 . [Online]
2. sysmagazine.com/posts/169713 . [Online]
3. ru.wikipedia.org/wiki/Katarsis . [Online]
4. databene.org/download/databene-benerator-manual-0.8.1.pdf . [Online]
5. dev.mysql.com/downloads/workbench . [On the web]
6. ajaxs.ru/lesson/javascript/137-transliteracija_stroki_na_javascript.html . [In the Internet]