September 12, 2014 at 16:17

Ejabberd 14.x on FreeBSD 10: Installation and Basic Configuration

    Good day!

    A small preface: not so long ago the 13th was released, and this year the 14th version of ejabberd is also following it. Process One decided to change the versioning scheme for several reasons and version 2.x is now considered obsolete. The new ejabberd has divided into two branches - ejabberd Community Server and ejabberd Business Edition and has begun to develop quite actively.

    In our case, FreeBSD was chosen as the system, since it is required to support a fairly large number of connections per node (~ 100k).

    I don’t see any reason to stop on installing the system, so under the cut, we’ll immediately proceed to the configuration.

    Ejabberd will store some of its data in mysql, so the odbc libraries must be installed on the system. We also use an external authorization script in php, so php is also needed.

    Stage 1:

    Set prerequisites (git, fop, libyaml, wget, unixODBC, expat, wx30-gtk2, php5, etc):

    pkg install git fop libyaml wget unixODBC-2.3.2 expat wx30-gtk2 php5 ... (+ все что вам нужно)
    cd /usr/ports/java/openjdk7/
    make install clean

    Download the latest version of Erlang (at the time of writing article 17.2.2): Build

    mkdir /usr/local/src && cd /usr/local/src
    git clone git://github.com/erlang/otp.git

    , put it:
    cd /usr/local/src/otp
    autoconf -f
    ./configure --prefix=/usr/local --enable-kernel-poll --with-ssl --enable-threads --with-odbc --enable-smp-support
    gmake
    gmake install

    Next, download ejabberd and ejabberd-contrib:
    cd /usr/local/src
    git clone git://github.com/processone/ejabberd.git
    git clone git://github.com/gamenet/ejabberd-contrib.git

    Create an ejabberd user, specify / var / lib / ejabberd as the home directory.

    We
    cd /usr/local/src/ejabberd
    autoconf -f
    ./configure --enable-mysql=yes --enable-odbc=yes --enable-lager=yes --prefix=/usr/local --localstatedir=/var --enable-user=ejabberd
    gmake && gmake install

    assemble and install ejabberd: During the assembly of erlang and ejabberd, various nuances can arise, otherwise you will need to create a number of symlinks from / usr / local / lib | include in / usr / lib | include or “play around” with LDFLAGS / CPPFLAGS.

    We compile mod_admin_extra to expand the list of available commands in ejabberdctl and via xmlrpc:
    cd /usr/local/src/ejabberd-contrib/mod_admin_extra
    ./build.sh
    cp ./ebin/mod_admin_extra.beam /usr/local/lib/ejabberd/ebin/

    Step 2:

    The main configuration file ejabberd.yml is replete with examples and comments of developers, so it’s not difficult to configure it. Tab is important in yaml-configuration files, the Habr editor ate it from me, so be careful! I will dwell only on some parameters:

    hosts:
    - "youserver.com"

    In the ejabberd_c2s section:
    backlog: 1024

    For external authorization:
    auth_method: external
    extauth_program: "/usr/local/bin/<ваш_скрипт>.php"
    extauth_instances: 12
    extauth_cache: 72000

    For connecting to mysql: We
    odbc_type: mysql
    odbc_server: "mysql_server_ip"
    odbc_database: "dbname"
    odbc_username: "username"
    odbc_password: "password"
    odbc_pool_size: 16
    odbc_keepalive_interval: 600

    specify the “admin”:
    admin:
    user:
    - "admin": "yourserver.com"

    Switch some modules to work with mysql:
    mod_last:
    db_type: odbc
    mod_muc:
    db_type: odbc
    mod_offline:
    db_type: odbc
    mod_privacy:
    db_type: odbc
    mod_private:
    db_type: odbc
    mod_roster:
    db_type: odbc
    mod_shared_roster:
    db_type: odbc
    mod_vcard:
    db_type: odbc

    Connect mod_admin_extra: We correct
    mod_admin_extra: {}

    ejabberdctl.cfg (the parameters are individual, but I quote the ones that are registered we have):
    ERL_MAX_PORTS=524288
    FIREWALL_WINDOW=4200-4210
    ERL_PROCESSES=5000000
    ERL_MAX_ETS_TABLES=262144
    ERLANG_NODE=ejabberd@node1

    We go to the mysql server, create a database there and fill it with a dump from mysql.sql from the sql folder of the ejabberd sources.

    Launch!
    su - ejabberd
    ejabberdctl start

    We create the admin user:
    ejabberdctl register admin yourserver.com password

    Step 3:
    To create a cluster, take another server and perform the procedures described above.
    For replication between nodes to work correctly (and indeed to assemble a cluster), it is necessary that the nodes can connect to each other on port 4369 and those ports that you specified in FIREWALL_WINDOW.

    We copy the file /var/lib/ejabberd/.erlang_cookie from the first node to the second and if there are no problems with the connections between the nodes, then we assemble the cluster:
    su - ejabberd
    erl -sname ejabberd@node2 -mnesia dir '"/var/lib/ejabberd/"' -mnesia extra_db_nodes "['ejabberd@node1']" -s mnesia
    Call mnesia: info ().
    If all is well, then in the list of running db nodes you will have both nodes.
    We change the table saving scheme, exit and run:
    mnesia:change_table_copy_type(schema, node(), disc_copies).
    q().
    ejabberdctl start

    Stage 4:
    A little about the system parameters:
    loader.conf
    kern.ipc.maxsockets=2400000.
kern.ipc.nmbclusters=0
net.inet.tcp.reass.maxsegments=2048
vm.pmap.shpgperproc=400
hw.em.rxd=4096
hw.em.txd=4096
hw.em.rx_int_delay=100
hw.em.tx_int_delay=100
hw.em.rx_abs_int_delay=1000
hw.em.tx_abs_int_delay=1000
dev.em.rx_processing_limit=-1
net.inet.tcp.hostcache.hashsize=4096
net.inet.tcp.hostcache.bucketlimit=100
net.inet.tcp.hostcache.cachelimit=65536
net.inet.tcp.syncache.hashsize=4096
net.inet.tcp.syncache.bucketlimit=120
net.inet.tcp.syncache.cachelimit=131072
net.inet.tcp.tcbhashsize=524288
net.isr.defaultqlimit=4096
net.isr.bindthreads=1
net.isr.maxthreads=4
net.link.ifqmaxlen=1024

    sysctl.conf
    kern.ipc.shm_use_phys=1
kern.ipc.somaxconn=8192
kern.maxfiles=3000000
kern.maxfilesperproc=2700000
kern.maxvnodes=256000
kern.random.sys.harvest.ethernet=0
kern.random.sys.harvest.interrupt=0
kern.sync_on_panic=1
net.inet.icmp.bmcastecho=0
net.inet.icmp.drop_redirect=1
net.inet.icmp.maskrepl=0
net.inet.ip.intr_queue_maxlen=256
net.inet.ip.maxfragpackets=1024
net.inet.ip.portrange.first=1024
net.inet.ip.portrange.last=65535
net.inet.ip.portrange.randomized=0
net.inet.ip.redirect=0
net.inet.ip.sourceroute=0
net.inet.ip.accept_sourceroute=0
net.inet.tcp.blackhole=2
net.inet.tcp.drop_synfin=1
net.inet.tcp.fast_finwait2_recycle=1
net.inet.tcp.finwait2_timeout=3000
net.inet.tcp.hostcache.expire=1200
net.inet.tcp.keepinit=5000
net.inet.tcp.maxtcptw=65536
net.inet.tcp.msl=5000
net.inet.tcp.recvbuf_auto=0
net.inet.tcp.recvspace=65536
net.inet.tcp.sendbuf_auto=0
net.inet.tcp.sendspace=131072
net.inet.tcp.syncookies=1
net.inet.tcp.tso=0
net.inet.udp.blackhole=1
net.inet.udp.recvspace=32768
net.isr.direct=1
net.route.netisr_maxqlen=1024
vfs.ufs.dirhash_maxmem=100000000



    If everything went fine, then going to the admin youserver.com:5280/admin/nodes/ in Running Nodes will be both nodes.

    mod_admin_extra greatly expands the list of commands that can be passed through ejabberdctl. If you left the ejabberd_xmlrpc module active (which has been built into ejabberd since version 13), then commands to the jabber server can also be transmitted via XmlRPC. We, for example, manage roster, groups, vcard'ami and other things through special workers written in php. But, in fact, you can connect to xmlrpc from anything.

    As for the consumed resources, for 100 thousand connected users, somewhere under 25-30 Gb of memory is required (this is when storing most data in mysql, i.e., purely for servicing connected clients). If you store data in mnesia, then the figure can easily reach 100 Gb or more, but it all depends on the number of users and the data in the database.

    Unfortunately, there are very few examples of successful ejabberd installations online that work with tens of thousands of connected users. Personally, I came across only “rumors” that the significantly patched ejabberd is the basis of WhatsApp services. Yes, and, in itself, erlang is a separate song, which can be difficult to find an approach :) ejabberd mailing lists are also half dead, and it’s problematic to consult with someone about possible problems on high-load, so a lot to learn with bitter experience. I am glad that ejabberd is now quite actively developing (compared with the period before the second half of 2013), a new functionality appears. This year, several major contributors from the community appeared who regularly correct bugs left over from 2.x versions or newly acquired.

    I will be glad to find "accomplices" who use ejabberd in their services - maybe we can productively share experience with each other!

    PS I would be grateful for pointing out the shortcomings (preferably in PM) and would be glad to any comments. Thanks!
    Tags:

    Also popular now: