November 8, 2012 at 18:45How to make Sphinx friends with OpenShift for ThinkingSphinx under Rails
I use Openshift to stage my small Rails projects. In principle, for small projects, it is very convenient - a convenient deployment, all the most necessary out of the box. What else might the soul need? But the soul wanted the sphinx, and at the same time she very much wanted to. Since I did not find a sphinx among the cartridges, I went to google for advice.
But nothing in Google ... Well, or almost nothing. Everyone advised raising DIY, rolling it all by hand, some of my friends advised me to transfer to AWS, but I didn’t want to admit that it’s impossible to raise the sphinx under OpenShift. And since the application was already spinning under the assembly for rails, I didn’t want to create DIY either, and I began to think how to raise sphinx in a ready-made environment.
Under the cut, what I came up with.
With rpm in OpenShift the same is tight, so we will put all of the sorts. We go to the sphinxsearch.com/downloads/release page , select “Source tarbal” and copy the link at the very bottom of the page (“ sphinxsearch.com/files/sphinx-2.0.6-release.tar.gz ”). My release was 2.0.6, so I will describe it for him.
Now go to your application under ssh:
we look at the “Git URL” parameter, from it we copy only the address with the UUID and go under it via ssh to the server.
Now go to the tmp directory, download and install spinx:
Pay attention to the prefix during configuration, it is necessary, otherwise sphinx will not be delivered.
Now we need to configure ThinkingSphinx (I assume that you have already configured Sphinx on the local machine, if not, then there is complete information about it, and also you have all the necessary files from OpenShift in the project). So the first thing you need to understand is that OpenShift will not allow us to bind Sphinx to localhost , the environment variable $ OPENSHIFT_INTERNAL_IP is used for all binders on OpenShift . It is also worth understanding that not all ports of Openshift will give you access for binding. Available ports range from 15,000 - 35530 . Therefore, we go to our c onfig / sphinx.yml and write the following:
I chose this port, you can choose your own, the main thing is that it lies within acceptable values.
There is one more limitation. OpenShift does not store log files in a standard rail directory , so we need to specify the paths for log files:
Since our searchd daemon is not in the standard folder , we need to register it in the config:
As a result, I got this kind of config:
Now we need to set the rules for restarting the sphinx, go to .openshift / action_hooks / deploy and write the following there:
This should build a new configuration file, rebuild the indexes, and restart the daemon.
However, our daemon now reassembles indexes only once during a deployment. To fix this we need to write a task for cron. I decided that it would be reassembled every minute, so go to .openshift / cron / minutely create a sphinx_rebuild file with the following contents:
That's it, now every minute sphinx will rebuild the index, so that your indexes will remain relevant for the current database.
All that was left for us was to shut down and enjoy the work of the sphinx on stage. By the way, all the same can be done on DIY and not necessarily for Rails.
But nothing in Google ... Well, or almost nothing. Everyone advised raising DIY, rolling it all by hand, some of my friends advised me to transfer to AWS, but I didn’t want to admit that it’s impossible to raise the sphinx under OpenShift. And since the application was already spinning under the assembly for rails, I didn’t want to create DIY either, and I began to think how to raise sphinx in a ready-made environment.
Under the cut, what I came up with.
Install Sphinx
With rpm in OpenShift the same is tight, so we will put all of the sorts. We go to the sphinxsearch.com/downloads/release page , select “Source tarbal” and copy the link at the very bottom of the page (“ sphinxsearch.com/files/sphinx-2.0.6-release.tar.gz ”). My release was 2.0.6, so I will describe it for him.
Now go to your application under ssh:
rhc app show -a [APP_NAME]
we look at the “Git URL” parameter, from it we copy only the address with the UUID and go under it via ssh to the server.
Now go to the tmp directory, download and install spinx:
cd $OPENSHIFT_TMP_DIR
wget http://sphinxsearch.com/files/sphinx-2.0.6-release.tar.gz
tar -zxfv sphinx-2.0.6-release.tar.gz
cd sphinx-2.0.6-release
./configure --prefix=$OPENSHIFT_RUNTIME_DIR
make install
Pay attention to the prefix during configuration, it is necessary, otherwise sphinx will not be delivered.
ThinkingSphinx Setup
Now we need to configure ThinkingSphinx (I assume that you have already configured Sphinx on the local machine, if not, then there is complete information about it, and also you have all the necessary files from OpenShift in the project). So the first thing you need to understand is that OpenShift will not allow us to bind Sphinx to localhost , the environment variable $ OPENSHIFT_INTERNAL_IP is used for all binders on OpenShift . It is also worth understanding that not all ports of Openshift will give you access for binding. Available ports range from 15,000 - 35530 . Therefore, we go to our c onfig / sphinx.yml and write the following:
production:
address: <%=ENV['OPENSHIFT_INTERNAL_IP']%>
port: 15000
I chose this port, you can choose your own, the main thing is that it lies within acceptable values.
There is one more limitation. OpenShift does not store log files in a standard rail directory , so we need to specify the paths for log files:
searchd_log_file: <%=File.join(ENV['OPENSHIFT_LOG_DIR'],'searchd.log')%>
query_log_file: <%=File.join(ENV['OPENSHIFT_LOG_DIR'],'searchd.query.log')%>
pid_file: <%=File.join(ENV['OPENSHIFT_LOG_DIR'],'searchd.production.pid')%>
Since our searchd daemon is not in the standard folder , we need to register it in the config:
bin_path: <%=File.join(ENV['OPENSHIFT_RUNTIME_DIR'],'bin')%>
searchd_binary_name: 'searchd'
indexer_binary_name: 'indexer'
As a result, I got this kind of config:
production:
searchd_log_file: <%=File.join(ENV['OPENSHIFT_LOG_DIR'],'searchd.log')%>
query_log_file: <%=File.join(ENV['OPENSHIFT_LOG_DIR'],'searchd.query.log')%>
pid_file: <%=File.join(ENV['OPENSHIFT_LOG_DIR'],'searchd.production.pid')%>
address: <%=ENV['OPENSHIFT_INTERNAL_IP']%>
port: 15000
bin_path: <%=File.join(ENV['OPENSHIFT_RUNTIME_DIR'],'bin')%>
searchd_binary_name: 'searchd'
indexer_binary_name: 'indexer'
Configure deployment
Now we need to set the rules for restarting the sphinx, go to .openshift / action_hooks / deploy and write the following there:
bundle exec rake ts:config RAILS_ENV="production"
bundle exec rake ts:rebuild RAILS_ENV="production"
This should build a new configuration file, rebuild the indexes, and restart the daemon.
However, our daemon now reassembles indexes only once during a deployment. To fix this we need to write a task for cron. I decided that it would be reassembled every minute, so go to .openshift / cron / minutely create a sphinx_rebuild file with the following contents:
!#/bin/bash
cd $OPENSHIFT_REPO_DIR
bundle exec rake ts:rebuild RAILS_ENV="production"
That's it, now every minute sphinx will rebuild the index, so that your indexes will remain relevant for the current database.
All that was left for us was to shut down and enjoy the work of the sphinx on stage. By the way, all the same can be done on DIY and not necessarily for Rails.