June 5, 2014 at 13:13

Ruby Integration in Nginx



    For a long time already there is a well-known bunch of Nginx + Lua, including a number of articles. But time does not stand still. About a year ago, the first version of the module integrating Ruby into Nginx appeared.

    Mruby


    For integration, not a full-fledged Ruby was chosen, but a subset of it, which is designed to be embedded in other applications, devices, etc. It has some limitations, but otherwise full Ruby. The project is called MRuby . Currently it already has version 1.0.0, i.e. considered stable.
    MRuby does not allow you to attach other files at run time, so the whole program must be in one file. At the same time, it is possible to convert the program to bytecode and execute it already, which has a positive effect on performance.
    Because there is no way to load other files, then existing gems are not suitable for it. To expand the functionality, its own format is used, which is both C code and Ruby in some places. These modules are assembled together with the library itself at compile time and are an integral part of it. There are binders to various databases for working with files, a network, and so on. A complete list is available on the site.
    Also there is a module that allows you to integrate this engine into Nginx, which was especially interesting.

    ngx_mruby


    So get to know : ngx_mruby . Module for connecting ruby ​​scripts to nginx. It has similar functionality with the Lua version. Allows you to perform operations at various stages of request processing.

    The module is assembled quite simply, the site has detailed instructions. Those who do not want to bother with the assembly can download the finished package:
    http://mruby.ajieks.ru/st/nginx_1.4.4-1~mruby~precise_amd64.deb
    MRuby in this assembly contains the following additional modules:

    As you can see, there is almost everything necessary for work. The only thing that this module did not find in the API was the ability to make a request outside. Most likely, it will need to be implemented as an extension and make a binding around the nginx API.

    The author shows a beautiful graph with tests, but did not find the configuration of the environment. Therefore, I just apply it for beauty:
    image

    Let's try to use


    So, the server is already installed. Everything is functioning, static is given. Add a bit of dynamics to this.
    As an example, I chose the task of parsing Markdown markup and rendering it in HTML without an additional server application. As well as line numbering in Ruby source.
    To do this, a clone of the sinatra repository is made and nginx is configured to solve the task.

    Markdown

    To process the markup, we will use the mruby-discount module connected to the assembly. It provides a simple class for working with markup. It is based on the C library of the same name, because the issue of performance, I think, will not particularly stand.
    To begin, write a program that will read the requested file from disk, process it and give it to the user.
    r = Nginx::Request.new
m = Discount.new("/st/style.css", "README")
filename = r.filename
filename = File.join(filename, 'README.md') if filename.end_with?('/')
markdown = File.exists?(filename) ? File.read(filename) : ''
Nginx.rputs m.header
Nginx.rputs m.md2html(markdown)
Nginx.rputs m.footer

    The first line is an instance of the request object containing all the necessary information, including the requested file, headers, URLs, URIs, etc.
    The next line creates an instance of the Discount class, indicating the style file and page title.
    This code does not handle 404 errors, so even if there is no file, there will always be a 200 return code.
    Now we connect it all
        location ~ \.md$ {
        add_header Content-Type text/html;
        mruby_content_handler "/opt/app/parse_md.rb" cache;
    }

    Result:
    mruby.ajieks.ru/sinatra
    mruby.ajieks.ru/sinatra/README.ru.md

    Ruby Files

    Initially, I planned to do not just numbering, but also coloring the code using the once-written code https://github.com/fuCtor/chalks . However, after all the adaptations made, problems arose in his work. The code seemed to work, but at some point it crashed with a Segmentation fault. The initial suspicion was a lack of memory allocated, but even after reducing its consumption, the problem did not disappear. After removing the code associated with the coloring, everything worked, but not as beautifully as I wanted.
    Change result
    module CGI
TABLE_FOR_ESCAPE_HTML__ = {"&"=>"&", '"'=>""", "<"=>"<", ">"=>">"}
def self.escapeHTML(string)
  string.gsub(/[&\"<>]/) do |ch|
  TABLE_FOR_ESCAPE_HTML__[ch]
  end
end
end
class String
  def ord
    self.bytes[0]
  end
end
class Chalk
  COMMENT_START_CHARS = {
      ruby: /#./,
      cpp: /\/\*|\/\//,
      c: /\/\//
  }
  COMMENT_END_CHARS = {
      cpp: /\*\/|.\n/,
      ruby: /.\n/,
      c: /.\n/,
  }
  STRING_SEP = %w(' ")
  SEPARATORS = " @(){}[],.:;\"\'`<>=+-*/\t\n\\?|&#"
  SEPARATORS_RX = /[@\(\)\{\}\[\],\.\:;"'`\<\>=\+\-\*\/\t\n\\\?\|\&#]/
  def initialize(file)
    @filename = file
    @file = File.new(file)
    @rnd = Random.new(file.hash)
    @tokens = {}
    reset
  end
  def parse &block
    reset()
    @file.read.each_char do |char|
      @last_couple = ((@last_couple.size < 2) ? @last_couple : @last_couple[1]) + char
      case(@state)
        when :source
          if start_comment?(@last_couple)
            @state = :comment
          elsif STRING_SEP.include?(char)
              @string_started_with = char
              @state = :string
          else
            process_entity(&block) if (@entity.length == 1 && SEPARATORS.index(@entity))  || SEPARATORS.index(char)
          end
        when :comment
          process_entity(:source, &block) if end_comment?(@last_couple)
        when :string
          if (STRING_SEP.include?(char) && @string_started_with == char)
            @entity += char
            process_entity(:source, &block)
            char = ''
          elsif char == '\\'
            @state = :escaped_char
          else
          end
        when :escaped_char
          @state = :string
      end
      @entity += char
    end
  end
  def to_html(&block)
    html = ''
    if block
      block.call( '
    ' )
    else
      html = '
    '
    end
    line_n = 1
    @file.readlines.each do
      if block
        block.call( "#{line_n}\n" )
      else
        html += "#{line_n}\n"
      end
      line_n += 1
    end
    @file = File.open(@filename)
    if block
      block.call( '
    ' )
    else
      html += '
    '
    end
    parse do |entity, type|
      entity = entity.gsub("\t", '  ')
      if block
        block.call( entity )
        #block.call(highlight( entity , type))
      else
        html += entity
        #html += highlight( entity , type)
      end
    end
    if block
      block.call( '
    ' )
    else
      html + '
    '
    end
  end
  def language
    @language ||= case(@file.path.to_s.split('.').last.to_sym)
      when :rb
        :ruby
      when :cpp, :hpp
        :cpp
      when  :c, :h
        :c
      when :py
        :python
      else
        @file.path.to_s.split('.').last.to_s
    end
  end
  private
  def process_entity(new_state = nil, &block)
    block.call @entity, @state if block
    @entity = ''
    @state = new_state if new_state
  end
  def reset
    @file = File.open(@filename) if @file
    @state = :source
    @string_started_with = ''
    @entity = ''
    @last_couple = ''
  end
  def color(entity)
    entity = entity.strip
    entity.gsub! SEPARATORS_RX, ''
    token = ''
    return token if entity.empty?
    #return token if token = @tokens[entity]
    return '' if entity[0].ord >= 128
    rgb = [ @rnd.rand(150) + 100, @rnd.rand(150) + 100, @rnd.rand(150) + 100 ]
    token = String.sprintf("#%02X%02X%02X", rgb[0], rgb[1], rgb[2])
    #token = "#%02X%02X%02X" % rgb
    #@tokens[entity] = token
    return token
  end
  def highlight(entity, type)
    esc_entity = CGI.escapeHTML( entity )
    case type
      when :string, :comment
        "#{esc_entity}"
      else
        rgb = color(entity)
        if rgb.empty?
          esc_entity
        else
          "#{esc_entity}"
        end
    end
  end
  def start_comment?(char)
    rx = COMMENT_START_CHARS[language]
    char.match rx if rx
  end
  def end_comment?(char)
    rx = COMMENT_END_CHARS[language]
    char.match rx if rx
  end
end


    And actually the code that performs the file reading and numbering:
    r = Nginx::Request.new
Nginx.rputs ''
begin
    ch = Chalk.new(r.filename)
    data = ch.to_html
    Nginx.rputs data
rescue => e
  Nginx.rputs e.message
end
Nginx.rputs ''

    We connect everything. Because the Chalk class is used constantly, we will load it in advance:
    mruby_init '/opt/app/init.rb';
    This line is added before the server section in the settings. Next, we already indicate our handler:
      location ~ \.rb$ {
        add_header Content-Type text/html;
        mruby_content_handler "/opt/app/parse_code.rb" cache;
   }

    That's it, now you can look at the result: mruby.ajieks.ru/sinatra/lib/sinatra/main.rb

    Conclusion


    Thus, it is possible to implement advanced query processing, filtering, caching, using another of the languages. Whether this module is ready for use in combat conditions, I do not know. While testing, there were freezes of the entire server, but there is a possibility of curvature of the hands, or still not everything is fully developed. I will follow the development of the project.

    Those interested can run the performance scripts indicated in the article using the links above.
    The server is deployed on DigitalOcean on the simplest machine, Ubuntu 12.04 x64. The number of processes 2, connections 1024. No additional settings were made. In the event of a server hang, I set a nginx reboot every 10 minutes.
    Tags:

    Also popular now: