Adding the ElasticSearch Apt Repository

[Lauren Finkel]

Can someone explain exactly how (as in step-by-step) I would input this step of the installation into the terminal?

2. Add the ElasticSearch apt repository next (fromhttp://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-repositories.html):

sudo wget -O - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add -

Then add this line to the bottom of /etc/apt/sources.list

deb http://packages.elasticsearch.org/elasticsearch/0.90/debian stable main

I think I put this in incorrectly and is what's causing my 500 internal server error when I try to access the dashboard.

Lauren

[Justin Simpson]

Hi Lauren,

I can try to help you here.  It sounds like elasticsearch did not get installed correctly, and that is what you think is causing your 500 internal server error message when you try to log into the dashboard.  It is possible that elasticsearch is installed, but not started, so I would verify that first.

Assuming you are logged into the ubuntu machine that Archivematica is installed on in a terminal, the following commands should do what you need to do.

copy and paste the following line, then hit enter:

aptitude show elasticsearch

That will take a moment, then spit out a bunch of output.  Look for a section like this:

Package: elasticsearch                  
State: installed
Automatically installed: no
Version: 0.90.13

If you don't see State: installed, then you need to install elasticsearch, which I will cover first. 

Installing ElasticSearch:

copy and paste the following line, then hit enter.  If you have done this before, it is still safe to do a 2nd time:

sudo wget -0 - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add -

now copy and paste this next line, and hit enter.

echo "deb http://packages.elasticsearch.org/elasticsearch/0.90/debian stable main" >> /etc/apt/sources.list

now you copy and paste each of these lines, hitting enter after each one:

sudo apt-get update

sudo apt-get install elasticsearch

sudo /etc/init.d/elasticsearch start

Now you should have elasticsearch installed and the dashboard should work.

If you already had elasticsearch installed, then doing these steps I have outlined won't hurt, but they won't help.  In that case you just need the last line:

sudo /etc/init.d/elasticsearch start

It is possible that you have installed elasticsearch, and started it, and it is still not running.  This can happen if you have a machine with less than the recommended amount of memory, for example.  One test to make sure elasticsearch is actually running is to run the following command:

curl http://localhost:9200

you should see a response back with "status": 200 and

Justin Simpson
Director of Archivematica Technical Services
www.artefactual.com
604-527-2056

--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematic...@googlegroups.com.
To post to this group, send email to archiv...@googlegroups.com.
Visit this group at http://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.

[Lauren Finkel]

Just for clarification I paste this line: echo "deb http://packages.elasticsearch.org/elasticsearch/0.90/debian stable main" >> /etc/apt/sources.lis AS IS into the terminal after the sudo wget line? I'm also assuming it's supposed to be sources.list, not .lis?

Lauren

[Justin Simpson]

Hi Lauren,

Yes, you paste that whole line as is, like so:

echo "deb http://packages.elasticsearch.org/elasticsearch/0.90/debian stable main" >> /etc/apt/sources.list

it is sources.list, yes. That command adds one line to the end fo the /etc/apt/sources.list file.  You could do this in a text editor as well, but I am suggesting this method because you asked for explicit step by step instructions and this is the simplest way to add one line to the end of a text file on a unix machine.

After you are done installing elasticsearch and you have started it, use the curl command to verify that elasticsearch is actually running.

How much memory have you allocated to the virtual machine that you are installing Archivematica in?  It is possible that ElasticSearch is not starting properly, or running out of memory, that is one possible reason why you are seeing the 500 error page.

Justin Simpson
Director of Archivematica Technical Services
www.artefactual.com
604-527-2056

--

[Lauren Finkel]

I ran the command sudo apt show elasticsearch and this is what I got:

I noticed that I'm running a different version than the one that's suggested on the site (I copied the link from the elasticsearch website instead of the archivematica instructions), would that have something to do with it?

I don't think memory should be an issue. I'm not running the program off a VM. I set my computer up to be a dedicated Linux machine, so it's running off a 300GB partitioned hard drive (the other 200GB is dedicated to the BitCurator modified version of Ubuntu).

What I'm having trouble understanding is that after I did a complete wipe of the hard drive this morning to re-install Ubuntu on both partitions, the installation on the non-BitCurator side worked perfectly. I was able to log into the Storage Service, set up my user/admin information and password, then use the localhost link on the instruction page to get into the Dashboard. There I processed 4 packages without a problem (aside from the Archival Storage tab not working - it showed the same 500 Internal Server Error). Then I logged out of the Dashboard and Storage Service. I can still log into the Storage Service, but not the Dashboard.

I just tried the two command lines you told me copy and paste into the Terminal, and this is what I got:

$ sudo wget -0 - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add -
wget: invalid option -- '0'
Usage: wget [OPTION]... [URL]...

Try `wget --help' for more options.
gpg: no valid OpenPGP data found.

Any idea what I should do?

Lauren

[Lauren Finkel]

Hi Justin,

I forgot to add that I did run the curl line and it showed "status" 200 and so on...

When I run the sudo apt show elasticsearch command, the last thing it says on there is that there are two additional records. Is this at all relevant?

Lauren

On Thursday, May 21, 2015 at 4:45:21 PM UTC-5, jsimpson wrote:

[Justin Simpson]

Hi Lauren,

If you are trying to install version 1.3.2 of Archivematica, then yes, having ElasticSearch 1.5 installed is going to be a problem.  Archivematica makes use of ElasticSearch in the archival storage tab, it is possible to get Archivematica to store aips, with ElasticSearch 1.5, but when you go to the archival storage tab, Archivematica queries ElasticSearch to get information about the aips, and that is where you would see a 500 error, if you have a version of ElasticSearch 1.0 or higher.

The next version of Archivematica, version 1.4.0, which will be ready very soon now, uses ElasticSearch 1.0 and up, but you are going to have to either try a qa version of Archivematica, or uninstall ElasticSearch and install the 0.90 version.

I am a bit confused about your comments about the 'non-BitCurator side'.  I think there is some information about the installation you are attempting that I am missing. 

About the wget error, it is not relevant to you, as it turns you.  You have ElasticSearch installed, it just turns out to be a version that does not work with the version of Archivematica you are trying to use. 

Justin Simpson
Director of Archivematica Technical Services
www.artefactual.com
604-527-2056

[Lauren Finkel]

Hi Justin,
Thanks for your reply. So at this point I have two options, I can either: 1) Wait for Archivematica 1.4 to be released (which will work with the ES version that I have installed on my computer), and then do a program update; or 2) Uninstall ES 1.5 and re-install the .90 version so it will work with Archivematica 1.3.2

If I choose the second option - to uninstall ES 1.5 and re-install .90, what do I have to do to this? Does this require physically deleting files and folder from  my directory or am i able to to re-enter the proper lines of code from Step 2:

sudo wget -O - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add -

echo "deb http://packages.elastic.co/elasticsearch/0.9/debian stable main" | sudo tee -a /etc/apt/sources.list

and are those the correct lines to be entered? - for a step that is this integral to the working of the front-end of the program, I would really recommend that the installation instructions be A LOT more clear.

I should add that my main concern is with LOGGING IN to the dashboard and not accessing the Archival Storage tab - I've already found a way to get around that issue by accessing the files through the storage service.

Thanks again,

Lauren

[Lauren Finkel]

This is the last paragraph of my /etc/apt/sources.list file. I can see the problem here is that I have three package links to ES (all bolded) and the one of the very bottom is the one I ACTUALLY need to make the dashboard work. Because I don't have the permissions for the folder, I can edit the file but cannot save over it. I'm assuming that if I somehow get rid of the first two ones (that I have in red) it should work? If I can't get rid of them, Is there a command to prioritize the use of the proper .90 ES version (bolded black) over the 1.5?

## This software is not part of Ubuntu, but is offered by third-party
## developers who want to ship their latest software.
deb http://extras.ubuntu.com/ubuntu trusty main
deb-src http://extras.ubuntu.com/ubuntu trusty main
deb http://packages.elastic.co/elasticsearch/1.5/debian stable main

deb http://packages.elastic.co/elasticsearch/0.9/debian stable main

deb http://packages.elasticsearch.org/elasticsearch/0.90/debian stable main

Is this the problem or am I just way off?

Lauren

[Justin Simpson]

Hi Lauren,

Good sleuthing, you are correct that having those 3 lines is a problem, but, unfortunately, editing that file is not going to be sufficient to get your Archivematica installation into working order.  You already have ElasticSearch 1.5 installed, and downgrading ElasticSearch, from 1.5 to version 0.90, requires some additional manual steps.  You can google for how to do that, but I suspect that trying to downgrade ElasticSearch is going to be a bit harder than you want to take on right now.

I think you have 3 options:

1) wipe out your vm, and start over - following the Archivematica installation instructions faithfully (i.e. do not install ElasticSearch using other instructions, just use the Archivematica guide).

2) upgrade your 1.3.2 Archivematica installation to the qa version.

3) wait for the 1.4.0 version of Archivematica to be released, and upgrade then.

1.4.0 should be available very soon - next week probably.  Although option 2 here might look interesting, personally I would recommend you go with option 1 or option 2. 

Archivematica is a complicated stack of software - there is a lot to learn even in getting it installed successfully.  I think you have gotten very close, I hope you don't give up here. 

As an aside, when you are trying to edit files that your user does not own (like the /etc/apt/sources.list)  you can use the sudo command to temporarily elevate your permissions.  So if you are using the gedit text editor (for example) you could type in a terminal window:

sudo gedit /etc/apt/sources.list

You will get prompted for your users password, and then gedit will open with the file displayed.  You will be able to edit the file and save your changes.

Justin Simpson
Director of Archivematica Technical Services
www.artefactual.com
604-527-2056

[Lauren Finkel]

Hi Justin,

My options are either 2 or 3. I mentioned in my last post that the computer I'm using is a dedicated Linux machine and I'm not running a VM. Wiping it would cause me to lose everything I have on the hard drive.

Re: #2: Is this the proper link to the qa version? https://launchpad.net/~archivematica/+archive/ubuntu/qa
and this is all the code I need:

deb http://ppa.launchpad.net/archivematica/qa/ubuntu trusty main 
deb-src http://ppa.launchpad.net/archivematica/qa/ubuntu trusty main

do I paste it (as is, one line at a time, using sudo before each command) into the terminal?
Is there anything I need to do or input before or after these two lines?

Or instead of using those two lines of code do I just follow the instructions that are posted on the "How Do I add Software from a ppa?" (https://launchpad.net/+help-soyuz/ppa-sources-list.html 

I'll also look into googling how to downgrade elasticsearch on Monday when I'm back at work.

Has there been any thought about offering the program as downloadable file that users can burn to a disk or usb instead (like BitCurator or Ubuntu?)

Thanks again for you help, I'm looking forward to being able to get through all of this soon.

Lauren

[Justin Simpson]

My options are either 2 or 3. I mentioned in my last post that the computer I'm using is a dedicated Linux machine and I'm not running a VM. Wiping it would cause me to lose everything I have on the hard drive.

ok, I thought you had said something about repartitioning and installing Archivematica in a separate vm from BitCurator. 

 

Re: #2: Is this the proper link to the qa version? https://launchpad.net/~archivematica/+archive/ubuntu/qa
and this is all the code I need:

Yes, that is the ppa for the qa version, but there is another ppa, externals-dev, that is required for the dependencies.

deb http://ppa.launchpad.net/archivematica/qa/ubuntu trusty main 
deb-src http://ppa.launchpad.net/archivematica/qa/ubuntu trusty main

do I paste it (as is, one line at a time, using sudo before each command) into the terminal?
Is there anything I need to do or input before or after these two lines?

Or instead of using those two lines of code do I just follow the instructions that are posted on the "How Do I add Software from a ppa?" (https://launchpad.net/+help-soyuz/ppa-sources-list.html 

I suggest you follow the instructions in that link, yes.  

I'll also look into googling how to downgrade elasticsearch on Monday when I'm back at work.

Has there been any thought about offering the program as downloadable file that users can burn to a disk or usb instead (like BitCurator or Ubuntu?)

Artefactual provides packages and maintains the github repositories for Archivematica, we don't have the internal resources to provide additional packaging work.  This is something that could be developed in the community, or Artefactual could be hired to do the work.