How to make friends with Ebean Gradle and make peace with IntelliJ Idea
I am finally ripe to start my web project. Another todo manager who aggregates tasks from the sources I need. It was planned as a project for the soul, so clean and correct. No compromises in architecture and technology. Only best-practices, only hardcore. And, of course, all this is going to push it in favorite Intellij IDEA.
After 7 years of Java, the last two mixed with Scala, I wanted to try Groovy. For assembly, of course, Gradle is popular and convenient. The rails seemed too “jaded”, so I decided to use Spring for the web, and in modern ways, through Spring Boot. And everything was just wonderful, only with ORM did not work out. At work, we sawed Hibernate, the customer personally disliked (do not laugh, and it happens - a separate story) and replaced it with our bike. Negative experience and unwillingness to pull a monster for a couple of entities did their job - hibernate is not hard! I wanted to try something completely different. By chance, I came across Ebean , which was chosen.
After the final stack selection, work began to boil. But bad luck, the cart with functionality has not yet budged. Under cat, a sincere excuse for why.
This opensource ORM framework, in my opinion, was created by haters of classic JPA implementations, just to be liked by hipsters. Of the key features:
That is, everything you need for a regular, non-enterprise web application. The authors of the framework did a tremendous job. I think it was not for nothing that it was added as one of the regular ORMs on Play! First, it scares that the site is somehow poorly updated, and the development froze. But no, on GitHub there are very recent versions of avaje-ebeanorm 4.x. And thanks to the community and the popularity of Play, the development of the project continues. As proof of activity on GitHub:
Here are some examples of what basic queries look like in Ebean:
Creating, saving and updating entities:
Work with partial objects:
Inspired by the examples, the final decision was made to use the latest, actively supported version 4.1.8. The fighting spirit. A dependency has been added to build.gradle:
Created configuration:
And the test entity:
I already rubbed my hands in anticipation of everyone's beloved profit. But ... there are no fairy tales, and of course, everything fell off at the start with java.lang.IllegalStateException: Bean class xxxTestEntity is not enhanced?
Usually people read documentation only when something fails. And that’s even good. It turns out that for the normal operation of Ebean, it is required to expand the bytecode of the .class files at the assembly stage, i.e. immediately after compilation. Why is this needed? Almost all ORMs are built into classes; most are simply twisted differently. Hibernate, for example, creates a runtime proxy through cglibto intercept access to Lazy collections. Transparent honest lazy without such hacks simply can not be done. Ebean, along with everyone, supports lazy loading, plus partial objects, and also tracks changes in each field so as not to send unnecessary data to the server during the save operation.
Early versions of the library supported two approaches to proxying: patching a .class file and instrumentation of class code when loading through ClassLoader (it was required to connect an agent at the start of the JVM). Over time, for ease of support, only the first option was left.
Hmm ... Difficult, but ... Do people somehow live with Ebean in the Play Framework? It turns out that the ORM itself comes with a separate ebeanorm-agent library, which can expand the byte code of the compiled .class file, and SBT, on which the Play rests, successfully uses it. Since in Play the code assembly is only internal, everything works like a clock. And no one probably guesses what is going on behind the scenes.
But the question is, is there such a feature for Gradle? There is definitely a plugin for Maven. But for Gradle, I found absolutely nothing (maybe I was looking badly). At first I was upset, and even thought of quitting this venture ... But at the last moment I got together and decided what would not be to finish the matter.
The most convenient way to add your own build tools to Gradle is to create the buildSrc module in the project root directory. The code from this module will be automatically available in all other build scripts (all options are described here ).
Next, create build.gradle inside the buildSrc directory :
Although the buildSrc approach does not require the creation of a plugin (you can just write and call code from a Groovy script), we will go the right way by expanding the Gradle API. After all, for sure, later, you will want to arrange it all as a finished product and put it out somewhere for general use.
The main idea of the plugin is to find and process the files created by the compiler that will be used by Ebean after each stage of compiling Java, Groovy or Scala. This problem is solved approximately like this:
Further, it is up to the expander itself. The algorithm is very simple: first, we recursively collect all .class files inside the base directory that are suitable for the filter, and then we pass them through the "agent". The processing itself is straightforward: there is an entity - a transformer, as well as a wrapper helper for processing from the input stream. After creating and linking both of us, it remains only to open the file and call transform (...), simultaneously placing a bunch of catch for possible errors. Everything in the assembly looks like this:
How filters are made, it makes no sense to show (or ashamed). It can be any implementation of the java.io.FileFilter interface. And in fact, this functionality is optional.
FileSystemClassBytesReader is another matter. This is a very important element of the process. It reads the associated .class files, if needed by the transformer. For example, when a subclass is analyzed, the ebean-agent requests a superclass through ClassBytesReader to check for the presence of the @MappedSuperclass annotation. Without this thing java.lang.IllegalStateException: Bean class xxxSubClassEntity is not enhanced? flies out without hesitation.
In order to call the plugin with the beautiful identifier 'ebean', you need to add the ebean.properties file to the buildSrc / resources / META-INF folder:
All. The plugin is ready.
And finally, add wonderful lines to the build.gradle of the main project:
Here is a story of a successful acquaintance of Ebean with Gradle. Everything is going and working as it should.
You can download the plugin on GitHub gradle-ebean-enhancer . Unfortunately, for now, everything is raw and the code needs to be copied to buildSrc. In the near future, finish and be sure to send to Maven Central and the Gradle repository.
Good news: There is a plugin for Yevgeny Krasik for Idea 13 , for which many thanks to him! The plugin “listens” to the build process and, in hot pursuit of the compiler, expands .class files. I need this in order to run and debug the Spring Boot application with Idea itself, because it is much more convenient and familiar.
The bad news: the plugin works with the old version of the agent library 3.2.2. The product of her activity is not compatible with Ebean 4.x and leads to strange actions at the start.
Solution: make a fork on github and rebuild the plugin for the desired version.
Everything went like clockwork. The application has started. The test entity was able to persist and load.
In fact, one could not write about it ... however, there is one “but”. Once I started building a hierarchy of entities and BaseEntity came up with @MappedSuperclass, java.lang.IllegalStateException: Bean class xxxSubClassEntity is not enhanced? here again.
javap showed that for some reason all subclasses are not extended. Why-yyyy? Why?
It turned out that an annoying error crept into the IDE plugin. During the process of expanding the current class, the transformer, as always, tries to analyze the subclass as well. To do this, I remind you that he needs to provide an implementation of ClassBytesReader. Only now, the implementation of the IDE plug-in for some reason, instead of binary data, "fed" the Groovy source code to the transformer, which he choked on.
So the fork turned out to be very helpful. It was:
It became:
Profit! I admit that the author of the plugin did not use much of the ORM framework itself. Although, the only thing I can complain about is the lack of sane error messages. After all, it is somehow sad to observe a quietly not working product.
The full, fixed plugin code can be found in the only idea-ebean-enhancer fork on GitHub at the time of writing . There is also a link to a zip ready for installation.
IntelliJ Idea users have finally got a fully working plugin, with support for the latest version of Ebean.
Along with Maven, Gradle also got an extension to support this ORM framework.
I hope that the path I have done will help readers dare to try what kind of beast this Ebean is . After all, it seems that all significant barriers along this path have been overcome. Well, or at least it will inspire you to go a similar way and enjoy digging into the insides of some unfamiliar library.
It also seemed pretty funny to me that writing code took significantly less time than preparing this publication.
After 7 years of Java, the last two mixed with Scala, I wanted to try Groovy. For assembly, of course, Gradle is popular and convenient. The rails seemed too “jaded”, so I decided to use Spring for the web, and in modern ways, through Spring Boot. And everything was just wonderful, only with ORM did not work out. At work, we sawed Hibernate, the customer personally disliked (do not laugh, and it happens - a separate story) and replaced it with our bike. Negative experience and unwillingness to pull a monster for a couple of entities did their job - hibernate is not hard! I wanted to try something completely different. By chance, I came across Ebean , which was chosen.
After the final stack selection, work began to boil. But bad luck, the cart with functionality has not yet budged. Under cat, a sincere excuse for why.
Ebean
This opensource ORM framework, in my opinion, was created by haters of classic JPA implementations, just to be liked by hipsters. Of the key features:
- familiar mapping (uses java.persistence annotations);
- simple API;
- easy to set up;
- flexible fetching related entities;
- partial sampling;
- tracking changes;
- lack of sessions;
- Native transaction support
- asynchronous loading;
- and even auto tuning!
That is, everything you need for a regular, non-enterprise web application. The authors of the framework did a tremendous job. I think it was not for nothing that it was added as one of the regular ORMs on Play! First, it scares that the site is somehow poorly updated, and the development froze. But no, on GitHub there are very recent versions of avaje-ebeanorm 4.x. And thanks to the community and the popularity of Play, the development of the project continues. As proof of activity on GitHub:
Here are some examples of what basic queries look like in Ebean:
// find an order by its id
Order order = Ebean.find(Order.class, 12);
// find all the orders
List list = Ebean.find(Order.class).findList();
// find all the orders shipped since a week ago
List list = Ebean.find(Order.class)
.where()
.eq("status", Order.Status.SHIPPED)
.gt("shipDate", lastWeek)
.findList();
Creating, saving and updating entities:
Order order = new Order();
order.setOrderDate(new Date());
...
// insert the order
Ebean.save(order);
//If the bean was fetched it will be updated
Order order = Ebean.find(Order.class, 12);
order.setStatus("SHIPPED");
...
// update the order
Ebean.save(order);
Work with partial objects:
// find order 12
// ... fetching the order id, orderDate and version property
// .... nb: the Id and version property are always fetched
Order order = Ebean.find(Order.class)
.select("orderDate")
.where().idEq(12)
.findUnique();
// shipDate is not in the partially populated order
// ... so it will lazy load all the missing properties
Date shipDate = order.getShipDate();
// similarly if we where to set the shipDate
// ... that would also trigger a lazy load
order.setShipDate(new Date());
Inspired by the examples, the final decision was made to use the latest, actively supported version 4.1.8. The fighting spirit. A dependency has been added to build.gradle:
compile('org.avaje.ebeanorm:avaje-ebeanorm:4.1.8')
Created configuration:
@Configuration
@PropertySource("config/db.properties")
class EbeanConfig {
@Autowired DbConfig dbConfig
@Bean
EbeanServer ebeanServer() {
EbeanServerFactory.create(serverConfig())
}
ServerConfig serverConfig() {
new ServerConfig(
dataSourceConfig: dataSourceConfig(), name: "main",
ddlGenerate: false, ddlRun: false, defaultServer: true, register: true,
namingConvention: new MatchingNamingConvention()
)
}
DataSourceConfig dataSourceConfig() {
new DataSourceConfig(driver: dbConfig.driver, username: dbConfig.username, password: dbConfig.password, url: dbConfig.url)
}
}
And the test entity:
@Entity
class TestEntity {
@Id UUID id
Integer value
}
I already rubbed my hands in anticipation of everyone's beloved profit. But ... there are no fairy tales, and of course, everything fell off at the start with java.lang.IllegalStateException: Bean class xxxTestEntity is not enhanced?
Usually people read documentation only when something fails. And that’s even good. It turns out that for the normal operation of Ebean, it is required to expand the bytecode of the .class files at the assembly stage, i.e. immediately after compilation. Why is this needed? Almost all ORMs are built into classes; most are simply twisted differently. Hibernate, for example, creates a runtime proxy through cglibto intercept access to Lazy collections. Transparent honest lazy without such hacks simply can not be done. Ebean, along with everyone, supports lazy loading, plus partial objects, and also tracks changes in each field so as not to send unnecessary data to the server during the save operation.
Early versions of the library supported two approaches to proxying: patching a .class file and instrumentation of class code when loading through ClassLoader (it was required to connect an agent at the start of the JVM). Over time, for ease of support, only the first option was left.
Hmm ... Difficult, but ... Do people somehow live with Ebean in the Play Framework? It turns out that the ORM itself comes with a separate ebeanorm-agent library, which can expand the byte code of the compiled .class file, and SBT, on which the Play rests, successfully uses it. Since in Play the code assembly is only internal, everything works like a clock. And no one probably guesses what is going on behind the scenes.
Gradle
But the question is, is there such a feature for Gradle? There is definitely a plugin for Maven. But for Gradle, I found absolutely nothing (maybe I was looking badly). At first I was upset, and even thought of quitting this venture ... But at the last moment I got together and decided what would not be to finish the matter.
So, make the missing plugin!
The most convenient way to add your own build tools to Gradle is to create the buildSrc module in the project root directory. The code from this module will be automatically available in all other build scripts (all options are described here ).
Next, create build.gradle inside the buildSrc directory :
apply plugin: 'groovy'
repositories {
mavenCentral()
}
dependencies {
//зависимость от самого свежего агента расширения
compile 'org.avaje.ebeanorm:avaje-ebeanorm-agent:4.1.5'
}
Although the buildSrc approach does not require the creation of a plugin (you can just write and call code from a Groovy script), we will go the right way by expanding the Gradle API. After all, for sure, later, you will want to arrange it all as a finished product and put it out somewhere for general use.
The main idea of the plugin is to find and process the files created by the compiler that will be used by Ebean after each stage of compiling Java, Groovy or Scala. This problem is solved approximately like this:
class EbeanPlugin implements Plugin {
//известные задачи компиляции - хардкод!
private static def supportedCompilerTasks = ['compileJava', 'compileGroovy', 'compileScala']
//это точка связки плагина с проектом, который собирается
void apply(Project project) {
//указываем имя и тип контейнера для настроек плагина
def params = project.extensions.create('ebean', EbeanPluginParams)
//вытягиваем все задачи, которые есть в проекте...
def tasks = project.tasks
//... и пытаемся навесить хук на каждую возможную задачу
supportedCompilerTasks.each { compileTask ->
tryHookCompilerTask(tasks, compileTask, params)
}
}
private static void tryHookCompilerTask(TaskContainer tasks, String taskName, EbeanPluginParams params) {
try {
def task = tasks.getByName(taskName)
//хук вешаем на событие после выполнения задачи, т.е. компиляции
task.doLast({ completedTask ->
//делаем полезную работу в контексте корневой папки вывода
enhanceTaskOutput(completedTask.outputs, params)
})
} catch (UnknownTaskException _) {
; //просто плагин не активирован
}
}
private static void enhanceTaskOutput(TaskOutputs taskOutputs, EbeanPluginParams params) {
//у задачи может быть несколько точек вывода, хотя такие пока не попадались
taskOutputs.files.each { outputDir ->
if (outputDir.isDirectory()) {
def classPath = outputDir.toPath()
//создаем фильтр, который на всякий случай сужать объем работы
def fileFilter = new EbeanFileFilter(classPath, params.include, params.exclude)
//и делаем непосредственно само расширение
new EBeanEnhancer(classPath, fileFilter).enhance()
}
}
}
}
//это модель свойств плагина, которые можно задавать в клиентском build.gradle
class EbeanPluginParams {
String[] include = []
String[] exclude = []
}
Further, it is up to the expander itself. The algorithm is very simple: first, we recursively collect all .class files inside the base directory that are suitable for the filter, and then we pass them through the "agent". The processing itself is straightforward: there is an entity - a transformer, as well as a wrapper helper for processing from the input stream. After creating and linking both of us, it remains only to open the file and call transform (...), simultaneously placing a bunch of catch for possible errors. Everything in the assembly looks like this:
class EBeanEnhancer {
private final Path classPath
private final FileFilter fileFilter
EBeanEnhancer(Path classPath) {
this(classPath, { file -> true })
}
EBeanEnhancer(Path classPath, FileFilter fileFilter) {
this.classPath = classPath
this.fileFilter = fileFilter
}
void enhance() {
collectClassFiles(classPath.toFile()).each { classFile ->
if (fileFilter.accept(classFile)) {
enhanceClassFile(classFile);
}
}
}
private void enhanceClassFile(File classFile) {
def transformer = new Transformer(new FileSystemClassBytesReader(classPath), "debug=" + 1);//0-9 -> none - all
def streamTransform = new InputStreamTransform(transformer, getClass().getClassLoader())
def className = ClassUtils.makeClassName(classPath, classFile);
try {
classFile.withInputStream { classInputStream ->
def enhancedClassData = streamTransform.transform(className, classInputStream)
if (null != enhancedClassData) { //transformer returns null when nothing was transformed
try {
classFile.withOutputStream { classOutputStream ->
classOutputStream << enhancedClassData
}
} catch (IOException e) {
throw new EbeanEnhancementException("Unable to store an enhanced class data back to file $classFile.name", e);
}
}
}
} catch (IOException e) {
throw new EbeanEnhancementException("Unable to read a class file $classFile.name for enhancement", e);
} catch (IllegalClassFormatException e) {
throw new EbeanEnhancementException("Unable to parse a class file $classFile.name while enhance", e);
}
}
private static List collectClassFiles(File dir) {
List classFiles = new ArrayList<>();
dir.listFiles().each { file ->
if (file.directory) {
classFiles.addAll(collectClassFiles(file));
} else {
if (file.name.endsWith(".class")) {
classFiles.add(file);
}
}
}
classFiles
}
}
How filters are made, it makes no sense to show (or ashamed). It can be any implementation of the java.io.FileFilter interface. And in fact, this functionality is optional.
FileSystemClassBytesReader is another matter. This is a very important element of the process. It reads the associated .class files, if needed by the transformer. For example, when a subclass is analyzed, the ebean-agent requests a superclass through ClassBytesReader to check for the presence of the @MappedSuperclass annotation. Without this thing java.lang.IllegalStateException: Bean class xxxSubClassEntity is not enhanced? flies out without hesitation.
class FileSystemClassBytesReader implements ClassBytesReader {
private final Path basePath;
FileSystemClassBytesReader(Path basePath) {
this.basePath = basePath;
}
@Override
byte[] getClassBytes(String className, ClassLoader classLoader) {
def classFilePath = basePath.resolve(className.replace(".", "/") + ".class");
def file = classFilePath.toFile()
def buffer = new byte[file.length()]
try {
file.withInputStream { classFileStream -> classFileStream.read(buffer) }
} catch (IOException e) {
throw new EbeanEnhancementException("Failed to load class '$className' at base path '$basePath'", e);
}
buffer
}
}
In order to call the plugin with the beautiful identifier 'ebean', you need to add the ebean.properties file to the buildSrc / resources / META-INF folder:
implementation-class=com.avaje.ebean.gradle.EbeanPlugin
All. The plugin is ready.
And finally, add wonderful lines to the build.gradle of the main project:
apply plugin: 'ebean' //имя property-файла добавленного в предыдущем шаге
ebean { //имя контейнера настроек плагина
include = ["com.vendor.product"]
exclude = ["SomeWeirdClass"]
}
Here is a story of a successful acquaintance of Ebean with Gradle. Everything is going and working as it should.
You can download the plugin on GitHub gradle-ebean-enhancer . Unfortunately, for now, everything is raw and the code needs to be copied to buildSrc. In the near future, finish and be sure to send to Maven Central and the Gradle repository.
IntelliJ Idea
Good news: There is a plugin for Yevgeny Krasik for Idea 13 , for which many thanks to him! The plugin “listens” to the build process and, in hot pursuit of the compiler, expands .class files. I need this in order to run and debug the Spring Boot application with Idea itself, because it is much more convenient and familiar.
The bad news: the plugin works with the old version of the agent library 3.2.2. The product of her activity is not compatible with Ebean 4.x and leads to strange actions at the start.
Solution: make a fork on github and rebuild the plugin for the desired version.
Everything went like clockwork. The application has started. The test entity was able to persist and load.
In fact, one could not write about it ... however, there is one “but”. Once I started building a hierarchy of entities and BaseEntity came up with @MappedSuperclass, java.lang.IllegalStateException: Bean class xxxSubClassEntity is not enhanced? here again.
javap showed that for some reason all subclasses are not extended. Why-yyyy? Why?
It turned out that an annoying error crept into the IDE plugin. During the process of expanding the current class, the transformer, as always, tries to analyze the subclass as well. To do this, I remind you that he needs to provide an implementation of ClassBytesReader. Only now, the implementation of the IDE plug-in for some reason, instead of binary data, "fed" the Groovy source code to the transformer, which he choked on.
So the fork turned out to be very helpful. It was:
//virtualFile оказался ссылкой на исходный код на Groovy
if (virtualFile == null) {
return null;
}
try {
return virtualFile.contentsToByteArray(); // o_O ?
} catch (IOException e) {
throw new RuntimeException(e);
}
It became:
if (virtualFile == null) {
compileContext.addMessage(CompilerMessageCategory.ERROR, "Unable to detect source file '" + className + "'. Not found in output directory", null, -1, -1);
return null;
}
final Module fileModule = compileContext.getModuleByFile(virtualFile);
final VirtualFile outputDirectory = compileContext.getModuleOutputDirectory(fileModule);
//ищем нужный файл в директории вывода
final VirtualFile compiledRequestedFile = outputDirectory.findFileByRelativePath(classNamePath + ".class");
if (null == compiledRequestedFile) {
compileContext.addMessage(CompilerMessageCategory.ERROR, "Class file for '" + className + "' is not found in output directory", null, -1, -1);
return null;
}
try {
return compiledRequestedFile.contentsToByteArray();
} catch (IOException e) {
throw new RuntimeException(e);
}
Profit! I admit that the author of the plugin did not use much of the ORM framework itself. Although, the only thing I can complain about is the lack of sane error messages. After all, it is somehow sad to observe a quietly not working product.
The full, fixed plugin code can be found in the only idea-ebean-enhancer fork on GitHub at the time of writing . There is also a link to a zip ready for installation.
Summary
IntelliJ Idea users have finally got a fully working plugin, with support for the latest version of Ebean.
Along with Maven, Gradle also got an extension to support this ORM framework.
I hope that the path I have done will help readers dare to try what kind of beast this Ebean is . After all, it seems that all significant barriers along this path have been overcome. Well, or at least it will inspire you to go a similar way and enjoy digging into the insides of some unfamiliar library.
It also seemed pretty funny to me that writing code took significantly less time than preparing this publication.