How To: Blacklight on Debian and Apache
Tom Kaz
Hi All. I am completely new to the Blacklight/Ruby/Rails/Solr/etc. technology stack, though I am somewhat familiar with Apache on Debian. I am also a one-man shop with limited programming resources. However, I am determined to implement Blacklight as our production discovery platform. We are currently using Encore and EDS on our iii (Sierra) system. My goal is to document every successful step so others may duplicate and learn.
As an offering to the community, my first post is this tutorial on getting Blacklight’s Quickstart/Demo running on Debian Jessie and Apache 2.4. By following it, users should be able to get the demo site up on WEBrick first, then run Blacklight on Apache/Fusion Passenger/Jetty. Note that I had to do 2 separate Blacklight installs to accomplish this.
If you see things wrong please comment. Admins: if there is a better place for this, please move it.
I will post my questions separately (and I have lots of ‘em). Thanks to all in advance.
Tom K.
Fordham Law Library
Installing Blacklight from the project website on Debian Jessie and Apache.
I started with a Debian Wheezy/Apache environment. I was able to install Blacklight and run it successfully using the testing Rails WEBrick server. However, when porting Blacklight to Apache 2.2 I got errors relating to a bug in 2.2 that is resolved in Apache 2.4. Upgrading Wheezy using Jessie repositories to get 2.4 is not recommended, so I decided to upgrade the kernel to Jessie (the latest), install Apache 2.4, then completely repeat the Blacklight install process. That is what’s documented here.
If upgrading to Jessie/Apache 2.4 be aware: the Apache config changes, requiring editing of its config file, documented below; and eventually Grub2 is installed, which will change the way Debian looks when it boots - noted below.
If you are installing Jessie fresh, the upgrade notes may still help you.
# commands are run as root.
$ commands are run as non-root user.
Upgrade from Wheezy to Jessie
/etc/apt/sources.list should only have one entry (comment out or delete the rest):
deb http://mirrors.kernel.org/debian jessie main contrib
# apt-get update
# apt-cache search linux-image
The kernel we want (for Intel processors) is:
linux-image-3.16.0-4-686-pae - Linux 3.16 for modern PCs
# apt-get install linux-image-3.16.0-4-686-pae
Answer y for yes
Once installed:
# shutdown –fr now
When your system comes up you should see 3.16 and 3.2 in the grub menu. Choose the first entry (the new 3.16 kernel).
Check/fix the system time:
# date
May be hours or a day off.
Restart NTP:
# ntpq –p
This shows the servers you are now using to keep time
# date
Should show the correct date/time now!
Configure Jessie repositories
Edit /etc/apt/sources.list to read:
###### Debian Main Repos
deb http://httpredir.debian.org/debian jessie main
deb-src http://httpredir.debian.org/debian jessie main
###### Debian Updates
deb http://httpredir.debian.org/debian jessie-updates main
deb-src http://httpredir.debian.org/debian jessie-updates main
deb http://security.debian.org/ jessie/updates main
deb-src http://security.debian.org/ jessie/updates main
Note: remove the kernel.org entry.
Update keyring to avoid “cannot authenticate” package errors:
# apt-get install debian-archive-keyring
If you get a no longer required message, do:
# apt-get autoremove
Type y to confirm
Upgrade (or install) Apache:
# apt-get update
# apt-get install apache2
Answer y to begin the download/install. This will take a while.
If you are prompted to keep your locally customized config file, keep it unless you want to start fresh.
When the install is done go to your server’s IP. You should see your default site or the Apache welcome page.
If upgrading from a previous version with a customized apache config file (/etc/apache2/sites-available/default), Apache appears to be misconfigured/broken (forbidden error when going to IP).
Fix:
Edit /etc/apache2/sites-available/000-default.conf
(Note the new config file).
In the web directory section:
Change ‘Options FollowSymLinks’ to:
Options Indexes FollowSymLinks Includes ExecCGI
Comment out ‘Order allow, deny’ and ‘allow from all’
Add the line:
Require all granted
Restart Apache
# /etc/init.d/apache2 restart
At this point, Apache, SSH and date should all be working.
Prep the environment. For Blacklight we need:
Java
RVM
Ruby 1.9 or greater
Rails 4 (or greater?)
Install Java
# file /sbin/init
This tells us if we are running 32- or 64-bit Debian. (I am 32-bit).
Download the Linux 32-bit (x86) jre tar.gz from here:
http://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html
Note: you must accept the license to download the file.
Copy it to /home/yourusername
(You can do this via wget yourfileURL if you put the tar.gz on a webserver, or by mapping using samba.)
Once the file is in your home directory, do:
# mkdir /usr/local/java
# cp -r jre-8u60-linux-i586.tar.gz /usr/local/java
Note: the name of the file may change if you download a later version.
# cd /usr/local/java
# chmod a+x jre-8u60-linux-i586.tar.gz
# tar xvzf jre-8u60-linux-i586.tar.gz
# ls –a
. .. jre1.8.0_60 jre-8u60-linux-i586.gz
# cd /etc
# cp profile profile.bak
# vi profile
Add this to the end of the file:
JAVA_HOME=/usr/local/java/jre1.8.0_60
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin
export JAVA_HOME
export PATH
Save and exit the file.
Do the following commands:
# update-alternatives --install "/usr/bin/java" "java" "/usr/local/java/jre1.8.0_60/bin/java" 1
# update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/local/java/jre1.8.0_60/bin/javaws" 1
# update-alternatives --set java /usr/local/java/jre1.8.0_60/bin/java
# update-alternatives --set javaws /usr/local/java/jre1.8.0_60/bin/javaws
# . /etc/profile [note the dot-space]
# java –version
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) Client VM (build 25.60-b23, mixed mode)
Now reboot the system:
# shutdown –r now
Log back in as root.
Update the System
Let’s update our system (doing upgrade first):
# apt-get update
# apt-get upgrade
If you upgraded your kernel this will probably be a lot of stuff. Type y for yes.
Continue (now doing dist-upgrade):
# apt-get update
# apt-get dist-upgrade
If you upgraded your kernel this will probably be a lot of stuff. Type y for yes.
If asked to keep any locally customized config files, keep them unless you want to start fresh with any of your packages.
Run the commands again!
# apt-get update
# apt-get dist-upgrade
This may return a message that no packages need upgrade, but that some need to be autoremoved
# apt-get autoremove
This shouldn’t break anything.
Note that grub will probably be upgraded to grub2 and your boot screen will look different. Your system should boot into the newest kernel by default. To select kernels you now have to arrow down, select advanced options and then you’ll see the kernel list.
For security reasons going forward, we want to do a single user install. Therefore we will not be executing commands as root, but as some user other than root (represented here by yourusername). You must create this account:
# adduser yourusername
Then we must allow yourusername to execute sudo:
# apt-get update
# apt-get install sudo
Confirm y to install
# adduser yourusername sudo
Now exit completely, and log back in to run as yourusername with sudo.
Install RVM, Ruby and Rails from the project website (not via apt-get):
First install the project public key:
$ gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
Install RVM, Ruby and Rails:
$ sudo \curl -sSL https://get.rvm.io | bash -s stable --rails
This will take a while. It will sit on different functions, especially “installing ri documentation for rails-4.2.4” but do not worry. Just let it do its thing.
Next make sure that Debian is aware of the RVM installation:
$ source /home/yourusername/.rvm/scripts/rvm
This should just return you to the prompt.
Now log out completely and log back in as yourusername.
Run some RVM commands to make sure it’s working:
$ rvm info
You should see system info, rvm version, ruby info and path.
$ rvm list
You should see rvm rubies, and the ruby version (2.2.1)
$ ruby --version (note that’s 2 hyphens)
Should return ruby 2.2.1p85
$ rails --version
Should return 4.2.4 (or whatever latest version you installed).
I. Install Blacklight to /projects/search_app (testing environment)
According to the Blacklight Project Quick Start instructions, Blacklight should run from /home/yourusername/projects/search_app
From /home/yourusername do:
$ mkdir projects
$ cd projects
$ rails new search_app -m https://raw.github.com/projectblacklight/blacklight/master/template.demo.rb
NOTE: I found that doing this step alone (easy way install) will not properly install Blacklight. Jetty won’t run properly and I get a “there exists no core with the name blacklight-core” error. Other steps are required:
When the command above is done, go into the new /search_app directory to continue:
$ cd search_app
Install required gems that do not install properly with the demo script (no js runtime, jetty won’t run):
This last one may take a while
Uncomment: gem 'therubyracer', platforms: :ruby
Once that’s done, do:
$ bundle install
This line is very important to install the gems into the “gemfile”.
$ rails generate blacklight:install --devise --marc --jettywrapper
This will take some time
Populate the database
$ bundle install
Now go to http://blacklight_server_IP:8983/solr/#/blacklight-core/query
(eg: http://192.168.1.2:8983/solr/#/blacklight-core/query)
You should Apache Solr running OK!
$ rake solr:marc:index_test_data
This should index the text marc records included with Blacklight. There will be Java io exception errors.
Start Blacklight using Rails’ WEBrick testing web server:
$ rails server --binding=blacklight_server_IP
This starts the blacklight server and specifies its static IP (the default bind is localhost). Note the 2 hyphens before “binding”. Remember to be in /home/yourusername/projects/search_app to start server!
You should see the default Blacklight installation running at http://blacklight_server_IP:3000/catalog
To stop the server:
If you are still in the shell window you used to start the
server, simply do
CTRL + C
If you have logged into a new shell window, you must find the process ID (PID) of the /home/yourusername ruby then kill it:
You should see something like:
5132 pts/1 Sl+ 0:08 /home/yourusername/.rvm/rubies/ruby-2.2.1/bin/ruby bi
Then do:
$ kill -9 5132
Stop jetty:
In /home/yourusername/projects/search_app do:
$ rake jetty:stop
At this point we have completed the demo install. Make sure WEBrick and jetty are stopped to continue.
II. Install PhusionPassenger to run Blacklight on Apache
I was not able to get Blacklight to run on PhusionPassenger from /projects/search_app. I found doing a Blacklight install to /home/yourusername/blacklight worked. This is what we will do going forward.
Install Blacklight to /home/yourusername/blacklight
From /home/yourusername do:
$ rails new blacklight -m https://raw.github.com/projectblacklight/blacklight/master/template.demo.rb
$ cd blacklight
Now do all the other steps in I. above from “Install required gems” onward. Once complete, make sure WEBrick and jetty are stopped.
Install PhusionPassenger
Install Passenger’s PGP key and https support
$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 561F9B9CAC40B2F7
$ sudo apt-get install -y apt-transport-https ca-certificates
Create a source file for the Passenger Jessie repository:
$ sudo sh -c 'echo deb https://oss-binaries.phusionpassenger.com/apt/passenger jessie main > /etc/apt/sources.list.d/passenger.list'
Note: it is crucial to install the version that matches your OS version. Install Wheezy Passenger on Jessie and you’ll break Apache.
Update system
Install Passenger packages
$ sudo apt-get install -y libapache2-mod-passenger
$ sudo apt-get install apache2-threaded-dev
Type y for yes.
Restart Apache
$ sudo
/etc/init.d/apache2 restart
Then validate
$ sudo passenger-config validate-install
Select both passenger and apache
Output should be:
* Checking whether this Passenger install is in PATH... ✓
* Checking whether there are no other Passenger installations... ✓
* Checking whether Apache is installed... ✓
* Checking whether the Passenger module is correctly configured in Apache... ✓
Everything looks good. :-)
$ export rvmsudo_secure_path=0
$ rvmsudo passenger-memory-stats
Show which Ruby interpreter blacklight is using:
$ passenger-config about ruby-command
Look for the Apache PassengerRuby path, likely:
/home/yourusername/.rvm/gems/ruby-2.2.1/wrappers/ruby
Tell Passenger to which Ruby to use:
$ rvm use ruby-2.2.1
$ bundle install --deployment --without development test
This may take a while
Configure /home/yourusername/blacklight/config/secrets.yml
$ bundle exec rake secret
Output will be a long string (key).
Copy the key and use it to replace <%=ENV["SECRET_KEY_BASE"]%> at
production:
secret_key_base: (your_key_goes_here)
Secure some files.
From /home/yourusername/blacklight do:
$ chmod 700 config db
$ chmod 600 config/database.yml config/secrets.yml
From /home/yourusername/blacklight do:
$ bundle exec rake assets:precompile db:migrate RAILS_ENV=production
Configure Apache to run Blacklight
For the sake of this tutorial, we’ll run blacklight on port 80 with no customized domain.
Create apache .conf file for blacklight
$ sudo touch /etc/apache2/sites-enabled/blacklight.conf
$ sudo vi /etc/apache2/sites-enabled/blacklight.conf
Put this in the file:
<VirtualHost blacklight_server_IP:80>
ServerName blacklight_server_IP
# Tell Apache and Passenger where your app's 'public' directory is and where to find ruby
DocumentRoot /home/yourusername/blacklight/public
PassengerRuby /home/yourusername/.rvm/gems/ruby-2.2.1/wrappers/ruby
# Relax Apache security settings
<Directory /home/yourusername/blacklight/public>
Allow from all
Options -MultiViews
# Uncomment this if you're on Apache >= 2.4 (we are):
Require all granted
</Directory>
</VirtualHost>
$ sudo apache2ctl restart
$ rake jetty:start
Note that at this point, jetty must be started manually for blacklight to work.
You should see blacklight running at your blacklight_server_ip.
Next step: get Jetty to run as a service…
I hope this helps.