November 26, 2018 at 11:48

In any incomprehensible situation - write scripts

    image

    Scripts are one of the most common ways to make an application more flexible, with the ability to fix something right on the go. Of course, this approach also has drawbacks; you must always remember the balance between flexibility and manageability. But in this article we will not discuss “generally” about the pros and cons of using scripts, we will consider practical ways to implement this approach, and also introduce a library that provides a convenient infrastructure for adding scripts to applications written in the Spring Framework.

    A few introductory words


    When you want to add the ability to change business logic in an application without recompilation and subsequent deployment, then scripts are one of the ways that comes to mind in the first place. Often, scripts appear not because it was intended, but because it happened. For example, in the specification there is a part of the logic that is not completely clear right now, but in order not to spend an extra couple of days (and sometimes longer) for analysis, you can make an extension point and call a script - a stub. And then, of course, this script will be rewritten when the requirements become clear.

    The method is not new, and its advantages and disadvantages are well known: flexibility - you can change the logic on a running application and save time on a re-install, but, on the other hand, scripts are more difficult to test, hence the possible problems with security, performance, etc.

    Those tricks that will be discussed later can be useful both to developers who already use scripts in their application, and to those who are just thinking about it.

    Nothing personal, just scripting


    With JSR-233, scripting in Java has become very simple. There are enough scripting engines based on this API (Nashorn, JRuby, Jython and some more), so adding a bit of scripting magic to your code is not a problem:

    Map parameters = createParametersMap();
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine scriptEngine = manager.getEngineByName("groovy");
Object result = scriptEngine.eval(script.getScriptAsString("discount.groovy"), 
new SimpleBindings(parameters));

    Obviously, if such a code is scattered throughout the application, then it will turn into something incomprehensible. And, of course, if you have more than one script call in your application, you need to create a separate class to work with them. Sometimes you can go even further and make special classes that will wrap calls evaluateGroovy()in regular typed Java methods. These methods will have a fairly uniform utility code, as in the example:

    public BigDecimal applyCustomerDiscount(Customer customer, BigDecimal orderAmount) {
  Map params = new HashMap<>();
  params.put("cust", customer);
  params.put("amount", orderAmount);
  return (BigDecimal)scripting.evalGroovy(getScriptSrc("discount.groovy"), params);
}

    This approach greatly increases transparency when calling scripts from application code - you can immediately see what parameters the script accepts, what type they are and what is returned. The main thing is not to forget to add to the code writing standards a ban on calling scripts not from typed methods!

    We pump up scripts


    Despite the fact that scripts are simple, if you have a lot of them and you use them intensively, there is a real chance to run into performance problems. For example, if you use a bunch of groovy templates to generate reports and you run them at the same time, sooner or later this will become one of the bottlenecks in application performance.
    Therefore, many frameworks make various add-ons over the standard API for improving the speed of work, caching, monitoring of execution, using different scripting languages ​​in one application, etc.

    For example, a rather ingenious scripting engine was made in CUBA that supports additional features, such as:

    1. Ability to write scripts in Java and Groovy
    2. Class cache in order not to recompile scripts
    3. JMX bin to control the engine

    All this, of course, improves performance and usability, but still the low-level engine remains low-level, and still you need to read the script text, pass parameters and call the API to execute the script. So you still need to do some kind of wrapper in each project to make development even more efficient.

    And it would be unfair not to mention GraalVM - an experimental engine that can run programs in different languages ​​(JVM and non-JVM) and allows you to insert modules in these languages into Java applications . I hope that Nashorn will go down in history sooner or later, and we will have the opportunity to write parts of the code in different languages ​​in one source. But this is only a dream.

    Spring Framework: an offer that's hard to refuse?


    Spring has built-in script execution support built on top of the JDK API. You org.springframework.scripting.*can find many useful classes in the package - all so that you can conveniently use the low-level API for scripting in your application.

    In addition, there is a higher level of support, it is described in detail in the documentation . In short - you need to make a class in a scripting language (for example, Groovy) and publish it as a bean via an XML description:


    Once a bean is published, it can be added to its classes using IoC. Spring provides automatic updating of the script when changing text in the file, you can hang aspects on methods, etc.

    It looks good, but you need to make “real” classes in order to publish them; you cannot write a regular function in a script. In addition, scripts can only be stored in the file system, to use the database you have to climb inside Spring. Yes, and many consider the XML configuration to be obsolete, especially if the application already has everything on the annotations. This, of course, is flavoring, but one often has to reckon with it.

    Scripts: Difficulties and Ideas


    So, each solution has its own price, and if we talk about scripts in Java applications, then when introducing this technology, one may encounter some difficulties:

    1. Controllability. Often, script calls are scattered throughout the application, and with changes in the code it is quite difficult to track the calls of the necessary scripts.
    2. Ability to find dial peers. If something goes wrong in a particular script, then finding all its dial peers will be a problem, unless you apply a search by file name or method calls likeevaluateGroovy()
    3. Transparency. Writing a script is not an easy task in itself, and even more difficult is for those who call this script. You need to remember what the input parameters are called, what type of data they have and what is the result of the execution. Or look at the script source code each time.
    4. Testing and updating - it is not always possible to test the script in the environment of the application code, and even after uploading it to the “battle” server, you need to somehow be able to quickly roll back everything if something goes wrong.

    It seems that wrapping script calls in Java methods will help solve most of the above problems. It is very good if such classes can be published in the IoC container and call methods with normal, meaningful names in their services, instead of being called eval(“disc_10_cl.groovy”)from some utility class. Another plus is that the code becomes self-documenting, the developer does not have to puzzle over what kind of algorithm is hidden behind the file name.

    On top of that, if each script will be associated with only one method, you can quickly find all the dial peers in the application using the “Find Usages” menu from the IDE and understand the place of the script in each specific business logic algorithm.

    Testing is simplified - it turns into “normal” class testing, using familiar frameworks, mocks, and more.

    All of the above is very consonant with the idea mentioned at the beginning of the article - “special” classes for methods that are implemented by scripts. But what if you take one more step and hide all the service code of the same type for calling script engines from the developer so that he doesn't even think about it (well, almost)?

    Script repositories - concept


    The idea is quite simple and should be familiar to those who at least once worked with Spring, especially with Spring JPA. What you need is to make a Java interface and call the script when calling its methods. In JPA, by the way, an identical approach is used - the call to CrudRepository is intercepted, based on the method name and parameters, a request is created, which is then executed by the database engine.

    What is needed to implement the concept?

    First, a class level annotation so that you can find the interface - the repository and make a bin based on it.

    Also, annotations on the methods of this interface will probably come in handy in order to store the metadata needed to call the method. For example - where to get the script text and which engine to use.

    A useful addition is the ability to use methods with implementation in the interface (aka default) - this code will work until the business analyst displays a more complete version of the algorithm, and the developer makes a script based on
    this information. Or let the analyst write the script, and the developer then simply copies it to the server. There are many options :-)

    So, suppose that for the online store you need to make a service to calculate discounts based on the user profile. It’s not clear right now how to do this, but the business analyst swears that all registered users are entitled to a 10% discount, he will find out the rest from the customer within a week. Service is needed right tomorrow - season after all. What might the code look like for this case?

    @ScriptRepository
public interface PricingRepository {
  @ScriptMethod
  default BigDecimal applyCustomerDiscount(Customer customer, BigDecimal orderAmount) {
      return orderAmount.multiply(new BigDecimal("0.9"));
  }
}

    And then the algorithm itself, written, for example, in groovy, will arrive in time, there the discounts will be slightly different:

    def age = 50
if ((Calendar.YEAR - customer.birthday.year) >= age) {
   return orderAmount.multiply(0.75)
} else {
   return orderAmount.multiply(0.9)
}

    The purpose of all this is to give the developer the ability to write only the interface code and script code, and not mess around with all these calls getEngine, evaland others. The library for working with scripts should do all the magic - intercept the invocation of the interface method, get the script text, substitute the parameter values, get the desired script engine, execute the script (or call the default method if there is no script text) and return the value. Ideally, in addition to the code that has already been written, the program should have something like this:

    @Service
public class CustomerServiceBean implements CustomerService {
   @Inject
   private PricingRepository pricingRepository;
   //Other injected beans here
   @Override
   public BigDecimal applyCustomerDiscount(Customer cust, BigDecimal orderAmnt) {
   if (customer.isRegistered()) {
       return pricingRepository.applyCustomerDiscount(cust, orderAmnt);
   } else {
       return orderAmnt;
   }
   //Other service methods here
 }

    The challenge is readable, understandable, and to make it, one does not need to have any special skills.

    These were the ideas on the basis of which a small library for working with scripts was made. It is intended for Spring applications, this framework was used to create the library. It provides an extensible API for loading scripts from various sources and executing them, which hides the routine work with script engines.

    How it works


    For all interfaces that are marked @ScriptRepository, proxy objects are created during the initialization of the Spring context using the newProxyInstanceclass method Proxy. These proxies are published in the Spring context as singleton beans, so you can declare a class field with an interface type and put an annotation on it @Autowiredor @Inject. Exactly as planned.

    Scanning and processing of script interfaces is activated with the help of annotation @EnableSсriptRepositories, just like in Spring, JPA or repositories for MongoDB ( @EnableJpaRepositoriesand @EnableMongoRepositoriesaccordingly) are activated . As annotation parameters, you need to specify an array with the names of the packages that you want to scan.

    @Configuration
@EnableScriptRepositories(basePackages = {"com.example", "com.sample"})
public class CoreConfig {
//More configuration here.
}

    Methods need to be marked with annotation @ScriptMethod(there is @GroovyScriptalso @JavaScriptone with the corresponding specialization) to add metadata for calling the script. Of course, default methods in interfaces are supported.

    The general structure of the library is shown in the diagram. Blue highlighted components that need to be developed, white - which are already in the library. The Spring icon marks components that are available in the Spring context.


    When the interface method is called (in fact, the proxy object), the call handler is launched, which in the application context searches for two beans: the provider, which will look for the script text, and the executor, which, in fact, the found text will execute. Then the handler returns the result to the calling method.

    The provider and executor bin names are indicated in the annotation @ScriptMethod, where you can also set a limit on the execution time of the method. Below is a sample library usage code:

    @ScriptRepository
public interface PricingRepository {
@ScriptMethod (providerBeanName = "resourceProvider", 
               evaluatorBeanName = "groovyEvaluator", 
               timeout = 100)
  default BigDecimal applyCustomerDiscount(
                     @ScriptParam("cust") Customer customer, 
                     @ScriptParam("amount") BigDecimal orderAmount) {
   return orderAmount.multiply(new BigDecimal("0.9"));
  }
}

    You can notice the annotations @ScriptParam- they are needed in order to indicate the names of the parameters when passing them to the script, since the Java compiler erases the original names from the sources (there are ways to make it not do this, but it's better not to rely on it). You can omit the parameter names, but in this case, you will need to use “arg0”, “arg1” in the script, which does not greatly improve readability.

    By default, the library has providers for reading .groovy and .js files from disk and corresponding executors, which are wrappers over the standard JSR-233 API. You can create your own beans for different script sources and for different engines, for this you need to implement the corresponding interfaces: ScriptProviderand SpringEvaluator. The first interface uses org.springframework.scripting.ScriptSourceand the second isorg.springframework.scripting.ScriptEvaluator. The Spring API was used so that ready-made classes could be used if they are already in the application.
    The provider and artist are searched by name for greater flexibility - you can replace the standard beans from the library in your application by naming your components with the same names.

    Testing and versioning


    Because scripts change frequently and easily, you need to have a way to make sure that the changes don't break anything. The library is compatible with JUnit, the repository can simply be tested as a regular class as part of a unit or integration test. Mock libraries are also supported, in the tests for the library you can find an example of how to make mock on the script repository method.

    If versioning is needed, then you can create a provider that will read different versions of scripts from the file system, from the database, or from Git, for example. So it will be easy to organize a rollback to the previous version of the script in case of problems on the main server.

    Total


    The presented library will help organize scripts in the Spring application:

    1. The developer will always have information about what parameters the scripts need and what is returned. And if the interface methods are named meaningfully, then what the script does.
    2. Providers and executors will help keep the code for receiving scripts and interacting with the script engine in one place and these calls will not be scattered throughout the application code.
    3. All script calls can be easily found using Find Usages.

    Spring Boot autoconfiguration, unit testing, mock are supported. You can get data about the “script” methods and their parameters through the API. And you can also wrap the execution result with a special ScriptResult object, in which there will be a result or an exception instance if you do not want to bother with try ... catch when invoking scripts. XML configuration is supported if it is required for one reason or another. And finally - you can specify a timeout for the script method, if the need arises.

    The library sources are here.
    Tags:

    Also popular now: