gotlium November 4, 2013 at 19:45

A universal way to create packages for various GNU Linux distributions

Tutorial

Various distributions provide their own utilities for building and installing programs.
For example, on Debian / GNU Linux, these are debuild and dpkg. On Red Hat Linux, rpmbuild and rpm.
Often we have to collect packages ourselves.
Installing programs through make, bypassing the package management system in distributions, is bad form.
In cases where we have the source code, and the author of the program took care of creating the Makefile, you can use checkinstall .
But it also happens that the author of a java application does not upload anything other than a jar file. Or we want to create some kind of package on our knees, and put it in the repository, so that in the future we can install / update it on many servers with different distributions. A utility called fpm will help us with this .
What is its merit? It out of the box allows us to collect packages for various distributions and even operating systems.

Now supported:

deb
rpm
solaris
tar
directories
Mac OS X .pkg files (osxpkg)

Installing the necessary packages for working with fpm on Debian GNU / Linux:

# apt-get install ruby rubygems rpm dpkg-dev
# gem install fpm

For example, let's compile the logstash package for Debian / GNU Linux and Red Hat Linux.
Create a build file in which we describe the dependencies, configuration files, and meta-information about the package:

$ mkdir logstash && cd logstash && touch build.sh && chmod +x build.sh && vim build.sh

Insert:

#!/bin/bash
JAR_URL="https://download.elasticsearch.org/logstash/logstash/logstash-1.2.2-flatjar.jar"
JAR_FILE="usr/share/logstash/logstash.jar"
DESCRIPTION="Logstash Open Source Log Management" 
EMAIL="admin@logstash.net"
URL='http://www.logstash.net/'
VERSION="1.2.2"
NAME="logstash"
# создаем структуру директорий
if [ ! -d "usr" ]; then
	mkdir -p {etc,usr/share/logstash,var/logstash}
fi
# скачиваем и кладем в необходимый каталог logstash
if [ ! -f "$JAR_FILE" ]; then
	wget "$JAR_URL" -O "$JAR_FILE"
fi
cd ..
function build() {
    fpm -n $NAME -v $VERSION -a all -C logstash -m "<$EMAIL>" \
        --pre-install logstash/preinstall \
        --description "$DESCRIPTION" \
        --url "$URL" -t "$1" -d "$2" \
        --config-files etc/logstash/syslog.conf \
        -s dir etc usr var
}
# задаем тип пакета и зависимости
build "deb" "default-jre"
build "rpm" "java-1.6.0-openjdk-devel"

Create the logstash configuration file:

$ mkdir -p etc/logstash && vim etc/logstash/syslog.conf

insert:

etc / logstash / syslog.conf

input {
  file {
    type => "syslog_file"
    exclude => [ "logstash.log*" ]
    path => [ "/var/log/messages", "/var/log/syslog", "/var/log/*.log" ]
  }
}
filter {
  grok {
    type => "syslog_relay"
    pattern => [ "^<[1-9]\d{0,2}>%{SPACE}%{GREEDYDATA:message_remainder}" ]
    add_tag => "got_syslog_pri"
    add_field => [ "syslog_raw_message", "%{@message}" ]
  }
  syslog_pri {
    type => "syslog_relay"
    tags => [ "got_syslog_pri" ]
  }
  mutate {
    type => "syslog_relay"
    tags => [ "got_syslog_pri" ]
    replace => [ "@message", "%{message_remainder}" ]
    remove => [ "message_remainder" ]
    remove_tag => "got_syslog_pri"
  }
  grok {
    type => "syslog_relay"
    pattern => [ "^%{SYSLOGTIMESTAMP:syslog_timestamp}%{SPACE}%{GREEDYDATA:message_remainder}" ]
    add_tag => "got_syslog_timestamp"
    add_field => [ "received_at", "%{@timestamp}" ]
  }
  date {
    type => "syslog_relay"
    tags => [ "got_syslog_timestamp" ]
    # season to taste for your own syslog format(s)
    match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss", "ISO8601" ]
  }
  mutate {
    type => "syslog_relay"
    tags => [ "got_syslog_timestamp" ]
    replace => [ "@message", "%{message_remainder}" ]
    remove => [ "message_remainder" ]
    remove_tag => "got_syslog_timestamp"
  }
  grok {
    type => "syslog_relay"
    pattern => [ "^%{SYSLOGHOST:syslog_hostname}%{SPACE}%{GREEDYDATA:message_remainder}" ]
    add_tag => "got_syslog_host"
    add_field => [ "logstash_source", "%{@source_host}" ]
  }
  mutate {
    type => "syslog_relay"
    tags => [ "got_syslog_host" ]
    replace => [ "@source_host", "%{syslog_hostname}" ]
    replace => [ "@message", "%{message_remainder}" ]
    remove => [ "message_remainder" ]
    remove_tag => "got_syslog_host"
  }
  grok {
    type => "syslog_relay"
    pattern => [ "^%{SYSLOGPROG:syslog_prog}:%{SPACE}%{GREEDYDATA:message_remainder}" ]
    add_tag => "got_syslog_prog"
  }
  mutate {
    type => "syslog_relay"
    tags => [ "got_syslog_prog" ]
    replace => [ "@message", "%{message_remainder}" ]
    remove => [ "message_remainder" ]
    remove_tag => "got_syslog_prog"
  }
  dns {
    type => 'syslog_relay'
    reverse => [ "@source_host", "@source_host" ]
    action => "replace"
  }
  mutate {
    type => "syslog_relay"
    tags => [ "syslog" ]
    remove => [ "syslog_hostname", "syslog_timestamp" ]
  }
}

Next, create a script that should be executed before installing the package:

$ cat > preinstall  << END
#!/bin/bash
useradd -g adm -r -m -d /usr/share/logstash -s /bin/false logstash || exit 0
END

Now just start the package build:

# ./build.sh

At the output, we get rpm and deb packages that are installed without problems in both Debian and Red Hat.

Install the package on the current machine and check:

# dpkg -i ../logstash*.deb && apt-get install -f
# java -jar /usr/share/logstash/logstash.jar agent -f /etc/logstash --log /var/log/logstash.log

Conclusion

In just a few commands, you can assemble a package for various distributions and OS in a couple of minutes.
Thanks to the author for such a wonderful utility. I hope someone finds her as useful as I do.

Tags:

A universal way to create packages for various GNU Linux distributions

Conclusion

Also popular now: