Extending Ruby with Ruby: borrowing function decorators from Python

From a translator: I suggest you translate the beginning of a presentation by Michael Fairley - Extending Ruby with Ruby . I translated only the first part of the three, because it has the maximum practical value and benefit, in my opinion. Nevertheless, I strongly recommend that you familiarize yourself with the full presentation, which in addition to Python provides examples of borrowing chips from Haskell and Scala.

Function Decorators

Python has such a thing - decorators, which is syntactic sugar for adding pieces of frequently used functionality to methods and functions. Now I will show you some examples of what decorators are and why they could be useful in Ruby.

I used to work a lot with Python, and function decorators are definitely something that I have been missing since then, and besides that can help almost all of us make our Ruby code cleaner.

Take Ruby and pretend that we need to transfer money from one bank account to another. Everything seems simple, right?

def send_money(from, to, amount)
  from.balance -= amount
  to.balance += amount
  from.save!
  to.save!
end

We deduct the amount from the balance of the “ from ” account ...

from.balance -= amount

And we add this amount to the balance of the “ to ” account ...

to.balance += amount

And save both accounts.

from.save!
to.save!

But there are a couple of shortcomings, the most obvious of which is the lack of a transaction (if “ from.save! ” Succeeds , but “ to.save! ” Does not, then the money will dissolve in the air).

Fortunately, ActiveRecord makes the solution to this problem very simple. We simply wrap our code in a transaction method block and this ensures that inside the block everything is completed either successfully or not.

def send_money(from, to, amount)
  ActiveRecord::Base.transaction do
    from.balance -= amount
    to.balance += amount
    from.save!
    to.save!
  end
end

Let's now look at the same example in Python. A version without a transaction looks almost exactly like in Ruby.

def send_money(from, to, amount):
  from.balance -= amount
  to.balance += amount
  from.save()
  to.save()

But it’s worth adding a transaction and the code starts to look not so elegant anymore.

def send_money(from, to, amount):
  try:
    db.start_transaction()
    from.balance -= amount
    to.balance += amount
    from.save()
    to.save()
    db.commit_transaction()
  except:
    db.rollback_transaction()
    raise

This method has 10 lines of code, but only 4 of them implement our business logic.

from.balance -= amount
to.balance += amount
from.save()
to.save()

The other 6 lines are a template for launching our logic inside a transaction. It's ugly and too verbose, but even worse - you have to remember all these lines, including the correct error handling and rollback semantics.

def send_money(from, to, amount):
  try:
    db.start_transaction()
    ...
    db.commit_transaction()
  except:
    db.rollback_transaction()
    raise

So how do we make it more beautiful and repeat ourselves less? There are no blocks in Python, so a focus like in Ruby will not work here. However, Python has the ability to easily pass and reassign methods. Therefore, we can write the “ transactional ” function , which will take another function as an argument and return the same function, but already wrapped in a template transaction code.

def send_money(from, to, amount):
  from.balance -= amount
  to.balance += amount
  from.save()
  to.save()
send_money = transactional(send_money)

And here is how the transactional function might look ...

def transactional(fn):
  def transactional_fn(*args):
    try:
      db.start_transaction()
      fn(*args)
      db.commit_transaction()
    except:
      db.rollback_transaction()
      raise
  return transactional_fn

It receives a function (" send_money " in our example) as its only argument.

def transactional(fn):

Defines a new function.

def transactional_fn(*args):

The new function contains a template for wrapping business logic in a transaction.

try:
  db.start_transaction()
  ...
  db.commit_transaction()
except:
  db.rollback_transaction()
  raise

Inside the template, the original function is called, which passes the arguments that were passed to the new function.

fn(*args)

Finally, the new feature is back.

return transactional_fn

Thus, we pass the send_money function to the transactional function that we just defined, which in turn returns a new function that does the same thing as the send_money function , but does it all inside the transaction. And then we assign this new function to our send_money function , overriding its original contents. Now, whenever we call the send_money function , the transaction version will be called.

send_money = transactional(send_money)

And this is what I have been leading all this time to. This idiom is so often used in Python that a special syntax, a function decorator, has been added to support it. And that’s how you do something transactional in Django ORM.

@transactional
def send_money(from, to, amount):
  from.balance -= amount
  to.balance += amount
  from.save()
  to.save()

So what?

Now you are thinking, “So what? You just showed how this decorating mumba-yumba solves the same problem that blocks solve. Why do we need this hat in Ruby? ” Well then, let's take a look at a case in which blocks no longer look so elegant.

Let us have a method that calculates the value of the nth element in the Fibonacci sequence.

def fib(n)
  if n <= 1
    1 
  else
    fib(n - 1) + fib(n - 2)
  end
end

He is slow, so we want to memoize him. The generally accepted approach for this is to shove " || = " everywhere , which suffers from the same ailment as the first example with a transaction - we mix the code of our algorithm with the additional behavior that we want to surround it with.

def fib(n)
  @fib ||= {}
  @fib[n] ||= if n <= 1
    1
  else
    fib(n - 1) + fib(n - 2)
  end
end

In addition, we forgot a couple of things here, such as the fact that “nil” and “false” cannot be memoized in this way: another point that must be constantly remembered.

def fib(n)
  @fib ||= {}
  return @fib[n] if @fib.has_key?(n)
  @fib[n] = if n <= 1
    1
  else
    fib(n - 1) + fib(n - 2)
  end
end

Well, we can solve this with a block, but the blocks do not have access to the name or arguments of the function that calls them, so we will have to pass this information explicitly.

def fib(n)
  memoize(:fib, n) do
    if n <= 1
      1
    else
      fib(n - 1) + fib(n - 2)
    end
  end
end

And now, if we start adding more blocks around the core functionality ...

def fib(n)
  memoize(:fib, n) do
    time(:fib, n) do
      if n <= 1
        1
      else
        fib(n - 1) + fib(n - 2)
      end
    end
  end
end

... we will be forced to retype the name of the method and its arguments again and again.

def fib(n)
  memoize(:fib, n) do
    time(:fib, n) do
      synchronize(:fib) do
        if n <= 1
          1
        else
          fib(n - 1) + fib(n - 2)
        end
      end
    end
  end
end

This is a rather fragile design and it will break the very moment we decide to change the method signature in any way.

Nevertheless, this can be solved by adding such a thing right after the definition of our method.

def fib(n)
  if n <= 1
    1
  else
    fib(n - 1) + fib(n - 2)
  end
end
ActiveSupport::Memoizable.memoize :fib

And this should remind you of what we saw in Python - when the modification of the method went immediately after the method itself.

# Ruby
def fib(n)
  ...
end
ActiveSupport::Memoizable.memoize :fib
# Python
def fib(n):
  ...
fib = memoize(fib)

Why didn't the Python community like this solution? Two reasons:

• You can no longer track your code from top to bottom;
• it's too easy to move the method somewhere and forget to do it with the code that came after it.

Let's look at our Fibonacci example in Python.

def fib(n):
  if n <= 1:
    return 1
  else
    return fib(n - 1) + fib(n - 2)

We want to memoize it, so we decorate it with the memoize function .

@memoize
def fib(n):
  if n <= 1:
    return 1
  else
    return fib(n - 1) + fib(n - 2)

And if we want to measure the running time of our method or synchronize its calls, then we just add another decorator. That's all.

@synchronize
@time
@memoize
def fib(n):
  if n <= 1:
    return 1
  else
    return fib(n - 1) + fib(n - 2)

And now I will show you how to achieve this in Ruby (using "+" instead of "@" and the first letter as a capital letter). And the funniest thing is that we can add this decorator syntax to Ruby, which is very close to the syntax in Python, with just 15 lines of code.

+Synchronized
+Timed
+Memoized
def fib(n)
  if n <= 1
    1
  else
    fib(n - 1) + fib(n - 2)
  end
end

Diving

Let's get back to our send_money example . We want to add the Transactional decorator to it .

+Transactional
def send_money(from, to, amount)
  from.balance -= amount
  to.balance += amount
  from.save!
  to.save!
end

« The Transactional » is a subclass of « the Decorator », which we will discuss below.

class Transactional < Decorator
  def call(orig, *args, &blk)
    ActiveRecord::Base.transaction do
      orig.call(*args, &blk)
    end
  end
end

He has only one call method , which will be called instead of our original method. As arguments, he receives a method that should “wrap”, his arguments and his block, which will be passed to him when called.

def call(orig, *args, &blk)

We open a transaction.

ActiveRecord::Base.transaction do

And then we call the original method inside the transaction block.

orig.call(*args, &blk)

Note that the structure of our decorator is different from the way decorators work in Python. Instead of defining a new function that will receive arguments, our Ruby decorator will receive the method itself and its arguments with every call. We are forced to do this because of the semantics of binding methods to objects in Ruby, which we will talk about below.

What is inside the class " Decorator "?

class Decorator
  def self.+@
    @@decorator = self.new
  end
  def self.decorator
    @@decorator
  end
  def self.clear_decorator
    @@decorator = nil
  end
end

This thing, “ + @ ”, is the “unary plus” operator, so this method will be called when we call + DecoratorName , as we did with + Transactional .

def self.+@

We also need a way to get the current decorator.

def self.decorator
    @@decorator
end

And a way to reset the current decorator.

def self.clear_decorator
    @@decorator = nil
end

A class that wants to have decorated methods should be extended by the MethodDecorators module .

class Bank
  extend MethodDecorators
  +Transactional
  def send_money(from, to, amount)
    from.balance -= amount
    to.balance += amount
    from.save!
    to.save!
  end
end

Could be expanded once the class « Class », but I think that the best practice in this case would leave the decision to the discretion of the end user.

module MethodDecorators
  def method_added(name)
    super
    decorator = Decorator.decorator
    return unless decorator
    Decorator.clear_decorator
    orig_method = instance_method(name)
    define_method(name) do |*args, &blk|
      m = orig_method.bind(self)
      decorator.call(m, *args, &blk)
    end
  end
end

“ Method_added ” is a private method of a class that is called every time a new method is defined in the class, giving us a convenient way to catch when the method was created.

def method_added(name)

Call the parent method_added . You can easily forget about this by overriding methods like " method_added ", " method_missing ", or " respond_to? "But if you do not, then you can easily break other libraries.

super

We get the current decorator and terminate the function if there is no decorator, otherwise we reset the current decorator. It’s important to reset the decorator, because then we redefine the method, which again calls our " method_added ".

decorator = Decorator.decorator
return unless decorator
Decorator.clear_decorator

We extract the original version of the method.

orig_method = instance_method(name)

And redefine it.

define_method(name) do |*args, &blk|

« Instance_method » actually returns an object of class « UnboundMethod », which is a method that does not know which object it belongs to, so we have to bind it to the current object.

m = orig_method.bind(self)

And then we call the decorator, passing it the original method and arguments for it.

decorator.call(m, *args, &blk)

What else?

Of course, there are a number of incredibly important points that must be resolved before this code can be considered ready for production.

Multiple decorators

The implementation that I cited allows us to use only one decorator for each method, but we want to be able to use more than one decorator.

+Timed
+Memoized
def fib(n)
  ...
end

Area of ​​visibility

« Define_method » defines the public methods, but we want private and protected methods, which could be decorated in accordance with their scope.

private
  +Transactional
  def send_money(from, to, amount)
    ...
  end

Class methods

“ Method_added ” and “ define_method ” only work for class instance methods, so you need to come up with something else so that decorators work for methods of the class itself.

+Memoize
def self.calculate
  ...
end

Arguments

In the Python example, I showed that we can pass values ​​to the decorator. We want us to be able to create any individual instances of decorators for our methods.

+Retry.new(3)
def post_to_facebook
  ...
end

gem install method_decorators

github.com/michaelfairley/method_decorators

I implemented all these features, added an exhaustive set of tests and rolled it all out in the form of gem. Use this because I think it can make your code cleaner, make it easier to read, and make it easier to maintain.