Merging Rails and Merb: Performance (Part 2 of 6)
- Transfer
Six consecutive articles on the merger of Rails and Merb were published on www.engineyard.com from December 2009 to April 2010. This is the second article. The first one is here .
The next great improvement we hoped to implement in Rails from Merb was better performance. Since Merb came after Rails, Merb developers had the opportunity to find out which parts of Rails were used more often and optimize their performance.
We wanted to take performance improvements from Merb and transfer them to Rails 3. In this post I will talk about several optimization points added in Rails 3: reducing the controller’s runtime and (strongly) accelerating the rendering of the partial collection.
To get started, we focused on the performance of several specific but widely used parts of Rails:
This is clearly an estimate, but it covered most cases where performance could be significantly better, but the Rails developer was not able to improve it on his own.
The first step was to improve the overhead of the Rails controller. There are no ways to test them in Rails 2.3, because you have to use
In doing this, we used Stefan Kaes' fork of ruby-prof, which comes with
When we looked at the process of the controller, it turned out that the dominant part was occupied by the creation of the answer. After delving deeper, we saw that the ActionController set the headers directly, then parsed them again before returning the response to get more information. A good example of this phenomenon is a header
As you can see, the Response object worked directly with the Content-Type header, steaming part of the header if necessary. This was especially problematic because Response did extra work on the headers during the preparations before sending the response to the client:
That is, before sending the response, Rails again broke the header
This did not take hundreds of milliseconds upon request, but in heavily cached applications, the costs could be higher than the cost of taking objects from the cache and returning them to the client.
The solution in this case was to store the content type and charset in the fields of the response object and combine them with one simple quick operation when preparing the response.
Now we just find the instance variables and create one String. Multiple changes to these lines of code have reduced the time of the eruptions from about 400 microseconds to 100 microseconds. Of course, not a very large amount of time, but it could really weaken performance-sensitive applications.
The rendering of the partial collection was another good opportunity for optimization. And in this case, the improvement was milliseconds, not microseconds!
To get started, the implementation in Rails 2.3:
An important part is what happened inside the cycle, which could happen hundreds of times in a large collection of partials. In this case, the implementation in Merb had more performance, which we were able to port to Rails. Here is the implementation of Merb.
Now we understand that this is far from ideal. A lot of things are happening here (and I personally would like to see this method refactored). But the interesting part is what happens inside the loop (starting from
For a collection of 100 partials, the difference between the costs could be about 10 ms and 3 ms. For the collection of small partials, this was noticeable (and the reason for the inline partials, which corresponded to the partials in the first place).
In Rails 3, we improved performance by reducing what happens inside the loop. Unfortunately, one feature of Rails made optimization a little harder. In particular, you could render a partial using a heterogeneous collection (a collection containing Post, Article and Page objects, for example) and Rails would render the correct template for each object (Article objects are rendered in
Faced with this problem, we were not able to fully optimize the heterogeneous case, but we did
Here's how the render of the collection now looks when we know the template:
What is important, the cycle itself is now small (even simpler than what happened inside the cycle in Merb). What is worth noting is that in the process of improving code performance, we created a PartialRenderer object to track state. Although you might think that creating a new object would be expensive, it turns out that creating objects in Ruby is relatively cheap and objects can offer caching capabilities that are much more complicated in procedural code.
For those who want to see improvements in pictures, here are a few things: firstly, an improvement between Rails 2.3 and Rails 3 on Ruby 1.9 (a smaller bar means faster speed).
And here for more expensive operations:
Finally, a comparison of Rails 3 on four platforms (Ruby 1.8, Ruby 1.9, Rubinius and JRuby):
As you can see, Rails 3 is noticeably faster than Rails 2.3, and that all platforms (including Rubinius!) Are noticeably improved over Ruby 1.8. All in all, a wonderful year for Ruby!
In the next post, I will talk about improvements in the Rails 3 API for plugin authors - keep an eye on messages and, as always, leave comments!
The next great improvement we hoped to implement in Rails from Merb was better performance. Since Merb came after Rails, Merb developers had the opportunity to find out which parts of Rails were used more often and optimize their performance.
We wanted to take performance improvements from Merb and transfer them to Rails 3. In this post I will talk about several optimization points added in Rails 3: reducing the controller’s runtime and (strongly) accelerating the rendering of the partial collection.
To get started, we focused on the performance of several specific but widely used parts of Rails:
- Overhead (router plus the cost of entry and exit from the controller)
- render: text
- render: template
- render: partial
- render several (10 and 100) identical partials in a loop
- render multiple (10 and 100) identical partial using collection
This is clearly an estimate, but it covered most cases where performance could be significantly better, but the Rails developer was not able to improve it on his own.
Controller overhead
The first step was to improve the overhead of the Rails controller. There are no ways to test them in Rails 2.3, because you have to use
render :stringit to send text to the client in response, which involves the entire rendering process. And yet, we wanted to reduce this as much as possible. In doing this, we used Stefan Kaes' fork of ruby-prof, which comes with
CallStackPrinter(the best way to visualize profile data from a Ruby application from all that I have seen). We also wrote several benchmarks that could duplicate the profiler in the process of its work, in order to focus on a specific site and get more accurate data.When we looked at the process of the controller, it turned out that the dominant part was occupied by the creation of the answer. After delving deeper, we saw that the ActionController set the headers directly, then parsed them again before returning the response to get more information. A good example of this phenomenon is a header
Content-Typethat has two components (the content-type itself and the optional charset). Both components were available in the Response object via getter and setter:Copy Source | Copy HTML
def content_type=(mime_type)
self.headers["Content-Type"] =
if mime_type =~ /charset/ || (c = charset).nil?
mime_type.to_s
else
"#{mime_type}; charset=#{c}"
end
end
# Returns the response's content MIME type, or nil if content type has been set.
def content_type
content_type = String(headers["Content-Type"] || headers["type"]).split(";")[ 0]
content_type.blank? ? nil : content_type
end
# Set the charset of the Content-Type header. Set to nil to remove it.
# If no content type is set, it defaults to HTML.
def charset=(charset)
headers["Content-Type"] =
if charset
"#{content_type || Mime::HTML}; charset=#{charset}"
else
content_type || Mime::HTML.to_s
end
end
def charset
charset = String(headers["Content-Type"] || headers["type"]).split(";")[1]
charset.blank? ? nil : charset.strip.split("=")[1]
end
As you can see, the Response object worked directly with the Content-Type header, steaming part of the header if necessary. This was especially problematic because Response did extra work on the headers during the preparations before sending the response to the client:
Copy Source | Copy HTML
def assign_default_content_type_and_charset!
self.content_type ||= Mime::HTML
self.charset ||= default_charset unless sending_file?
end
That is, before sending the response, Rails again broke the header
Content-Typeby a semicolon and then did more work with strings to connect them together again. And of course, it was Response#content_type=used in other parts of Rails to correctly set it depending on the type of template or inside blocks respond_to. This did not take hundreds of milliseconds upon request, but in heavily cached applications, the costs could be higher than the cost of taking objects from the cache and returning them to the client.
The solution in this case was to store the content type and charset in the fields of the response object and combine them with one simple quick operation when preparing the response.
Copy Source | Copy HTML
attr_accessor :charset, :content_type
def assign_default_content_type_and_charset!
return if headers[CONTENT_TYPE].present?
@content_type ||= Mime::HTML
@charset ||= self.class.default_charset
type = @content_type.to_s.dup
type < < "; charset=#{@charset}" unless @sending_file
headers[CONTENT_TYPE] = type
end
Now we just find the instance variables and create one String. Multiple changes to these lines of code have reduced the time of the eruptions from about 400 microseconds to 100 microseconds. Of course, not a very large amount of time, but it could really weaken performance-sensitive applications.
Partial collection render
The rendering of the partial collection was another good opportunity for optimization. And in this case, the improvement was milliseconds, not microseconds!
To get started, the implementation in Rails 2.3:
Copy Source | Copy HTML
def render_partial_collection(options = {}) #:nodoc:
return nil if options[:collection].blank?
partial = options[:partial]
spacer = options[:spacer_template] ? render(:partial => options[:spacer_template]) : ''
local_assigns = options[:locals] ? options[:locals].clone : {}
as = options[:as]
index = 0
options[:collection].map do |object|
_partial_path ||= partial ||
ActionController::RecordIdentifier.partial_path(object, controller.class.controller_path)
template = _pick_partial_template(_partial_path)
local_assigns[template.counter_name] = index
result = template.render_partial(self, object, local_assigns.dup, as)
index += 1
result
end.join(spacer).html_safe!
end
An important part is what happened inside the cycle, which could happen hundreds of times in a large collection of partials. In this case, the implementation in Merb had more performance, which we were able to port to Rails. Here is the implementation of Merb.
Copy Source | Copy HTML
with = [opts.delete(:with)].flatten
as = (opts.delete(:as) || template.match(%r[(?:.*/)?_([^\./]*)])[1]).to_sym
# Ensure that as is in the locals hash even if it isn't passed in here
# so that it's included in the preamble.
locals = opts.merge(:collection_index => -1, :collection_size => with.size, as => opts[as])
template_method, template_location = _template_for(
template,
opts.delete(:format) || content_type,
kontroller,
template_path,
locals.keys)
# this handles an edge-case where the name of the partial is _foo.* and your opts
# have :foo as a key.
named_local = opts.key?(as)
sent_template = with.map do |temp|
locals[as] = temp unless named_local
if template_method && self.respond_to?(template_method)
locals[:collection_index] += 1
send(template_method, locals)
else
raise TemplateNotFound, "Could not find template at #{template_location}.*"
end
end.join
sent_template
Now we understand that this is far from ideal. A lot of things are happening here (and I personally would like to see this method refactored). But the interesting part is what happens inside the loop (starting from
sent_template = with.map). Unlike ActionView, which looked up the template name, took the template object, counter name, etc., Merb limited the activity inside the loop by setting several Hash values and calling the method. For a collection of 100 partials, the difference between the costs could be about 10 ms and 3 ms. For the collection of small partials, this was noticeable (and the reason for the inline partials, which corresponded to the partials in the first place).
In Rails 3, we improved performance by reducing what happens inside the loop. Unfortunately, one feature of Rails made optimization a little harder. In particular, you could render a partial using a heterogeneous collection (a collection containing Post, Article and Page objects, for example) and Rails would render the correct template for each object (Article objects are rendered in
_article.html.erb, etc.). This means that it is not always possible to pinpoint a template that needs to be rendered. Faced with this problem, we were not able to fully optimize the heterogeneous case, but we did
render :partial => "name", :collection => @arrayfaster. To achieve this, we split the logic into 2 ways: faster for the case when we know the template, and slower for the case when it needs to be defined depending on the object. Here's how the render of the collection now looks when we know the template:
Copy Source | Copy HTML
def collection_with_template(template = @template)
segments, locals, as = [], @locals, @options[:as] || template.variable_name
counter_name = template.counter_name
locals[counter_name] = -1
@collection.each do |object|
locals[counter_name] += 1
locals[as] = object
segments < < template.render(@view, locals)
end
@template = template
segments
end
What is important, the cycle itself is now small (even simpler than what happened inside the cycle in Merb). What is worth noting is that in the process of improving code performance, we created a PartialRenderer object to track state. Although you might think that creating a new object would be expensive, it turns out that creating objects in Ruby is relatively cheap and objects can offer caching capabilities that are much more complicated in procedural code.
For those who want to see improvements in pictures, here are a few things: firstly, an improvement between Rails 2.3 and Rails 3 on Ruby 1.9 (a smaller bar means faster speed).
And here for more expensive operations:
Finally, a comparison of Rails 3 on four platforms (Ruby 1.8, Ruby 1.9, Rubinius and JRuby):
As you can see, Rails 3 is noticeably faster than Rails 2.3, and that all platforms (including Rubinius!) Are noticeably improved over Ruby 1.8. All in all, a wonderful year for Ruby!
In the next post, I will talk about improvements in the Rails 3 API for plugin authors - keep an eye on messages and, as always, leave comments!
