A How-To Solr installation on Ubuntu servers

[Mohammad S. AlMutairi]

A lot of newcomers who want to try DSpace specially non-technical people do face an issue installing Solr for DSpace. The DSpace Solr installation portion doesn't cover specific details about any Linux OS so to make things easier for the folks who are using Ubuntu I'm posting a detailed instructions how Solr should be installed on Ubuntu in a hope someone who deserve helping save his time and get Solr up and running in no time. See Solr installation steps and also the removal of solr if you ever need to remove it below. Hope it doesn't fire back as it did not long time ago!.

##################################### Solr Installation #####################################
# set a password for the root user.
sudo passwd root

# login with root to start solr installation.
su - root

a) mkdir /build
b) cd /build
c) wget https://downloads.apache.org/lucene/solr/8.11.1/solr-8.11.1.tgz
d) tar xzf solr-8.11.1.tgz solr-8.11.1/bin/install_solr_service.sh --strip-components=2
e) perl -i -pe 's/SOLR_USER=solr/SOLR_USER=tomcat/;' /build/install_solr_service.sh
f) bash ./install_solr_service.sh solr-8.11.1.tgz -f
g) echo SOLR_OPTS=\"\$SOLR_OPTS -Dsolr.allowPaths=/opt/dspace/solr/statistics,/opt/dspace/temp/solr-data\" >> /etc/default/solr.in.sh
h) cp -r /opt/dspace/solr/* /var/solr/data/ # Do this step after installing dspace backend (REST API server). You need to change /opt/dspace to the folder you installed dspace backend into.
i) chown -R tomcat:tomcat /opt/sol*
j) chown -R tomcat:tomcat /var/solr/data/
k) systemctl enable solr
l) systemctl restart solr

# Run curl as you see it below to test Solr and check the status of dspace cores you copied in step (h) above. Dspace cores names you should see and see it's data are (authority, oai, search and statistics).
curl http://localhost:8983/solr/admin/cores
curl http://localhost:8983/solr/admin/cores?action=STATUS

################################# End of Solr Installation ##################################

######################### Steps to manually uninstall Solr from Ubuntu ######################
# You need to login with root.
# login with root to remove old solr installation from your server.
su - root

1) systemctl stop solr
2) rm -r /var/solr
3) rm -r /opt/sol*
4) rm /etc/init.d/solr
5) deluser --remove-home solr
6) deluser --group solr
7) update-rc.d -f solr remove
8) rm -rf /etc/default/solr.in.sh
############################# End of Solr Removal instructions ##############################

"When the sage points at the moon, the fool looks at the finger"

[Chris Clawson]

Thanks for revisiting this! There is detail here which I have never seen, especially step e) . I will probably attempt a removal/re-installation of Solr in a few hours. Here is the results of my status checks, using curl:

root@media:~# curl http://localhost:8983/solr/admin/cores
{
  "responseHeader":{
    "status":0,
    "QTime":70},
  "initFailures":{},
  "status":{}}
root@media:~# curl http://localhost:8983/solr/admin/cores?action=STATUS
{
  "responseHeader":{
    "status":0,
    "QTime":1},
  "initFailures":{},
  "status":{}}
root@media:~#

... I don't see any DSpace names mentioned in these returns, so I am guessing there is an issue here.

[Mohammad S. AlMutairi]

Hello Chris,

Your solr installation is broken so you really really really must remove the old installation and begin a fresh install. All the provided instructions is very simple and easy to follow so just follow it. Regarding step (e) it just another and easier way of changing SOLR_USER=solr to SOLR_USER=tomcat using perl substitution. Don't stop at it or the (g) step just remove the old solr and install solr following the installation steps above but you MUST BE ROOT during the removal or the installing of Solr to overcome any permission issues you might confront.

#### Here is what you suppose to see if Solr and dspace cores are done correctly. This is part of it.#####

    "search":{
      "name":"search",
      "instanceDir":"/var/solr/data/search",
      "dataDir":"/var/solr/data/search/data/",
      "config":"solrconfig.xml",
      "schema":"schema.xml",
      "startTime":"2021-12-28T10:55:06.841Z",
      "uptime":11865277,
      "index":{
        "numDocs":45760,
        "maxDoc":45760,
        "deletedDocs":0,
        "indexHeapUsageBytes":489928,
        "version":678,
        "segmentCount":22,
        "current":true,
        "hasDeletions":false,
        "directory":"org.apache.lucene.store.NRTCachingDirectory:NRTCachingDirectory(MMapDirectory@/var/solr/data/search/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@12cd8c11; maxCacheMB=48.0 maxMergeSizeMB=4.0)",
        "segmentsFile":"segments_2z",
        "segmentsFileSizeInBytes":1947,
        "userData":{
          "commitCommandVer":"0",
          "commitTimeMSec":"1640647856055"},
        "lastModified":"2021-12-27T23:30:56.055Z",
        "sizeInBytes":1641346285,
        "size":"1.53 GB"}},
    "statistics":{
      "name":"statistics",
      "instanceDir":"/var/solr/data/statistics",
      "dataDir":"/var/solr/data/statistics/data/",
      "config":"solrconfig.xml",
      "schema":"schema.xml",
      "startTime":"2021-12-28T10:55:07.565Z",
      "uptime":11864565,
      "index":{
        "numDocs":78,
        "maxDoc":78,
        "deletedDocs":0,
        "indexHeapUsageBytes":38772,
        "version":46,
        "segmentCount":11,
        "current":false,
        "hasDeletions":false,
        "directory":"org.apache.lucene.store.NRTCachingDirectory:NRTCachingDirectory(MMapDirectory@/var/solr/data/statistics/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@12cd8c11; maxCacheMB=48.0 maxMergeSizeMB=4.0)",

[Chris Clawson]

I believe this has happened before... Problems begin with step 'f'. The following is the output from the bash command:

root@media:/build# bash ./install_solr_service.sh solr-8.11.1.tgz -f
Extracting solr-8.11.1.tgz to /opt
Installing symlink /opt/solr -> /opt/solr-8.11.1 ...
Installing /etc/init.d/solr script ...
Installing /etc/default/solr.in.sh ...
Service solr installed.
Customize Solr startup configuration in /etc/default/solr.in.sh
Job for solr.service failed because the control process exited with error code.
See "systemctl status solr.service" and "journalctl -xe" for details.
● solr.service - LSB: Controls Apache Solr as a Service
   Loaded: loaded (/etc/init.d/solr; generated)
   Active: failed (Result: exit-code) since Tue 2021-12-28 11:00:48 CST; 5s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1474 ExecStart=/etc/init.d/solr start (code=exited, status=1/FAILURE)

Dec 28 11:00:48 media systemd[1]: Starting LSB: Controls Apache Solr as a Service...
Dec 28 11:00:48 media su[1476]: Successful su for tomcat by root
Dec 28 11:00:48 media su[1476]: + ??? root:tomcat
Dec 28 11:00:48 media su[1476]: pam_unix(su:session): session opened for user tomcat by (uid=0)
Dec 28 11:00:48 media su[1476]: pam_unix(su:session): session closed for user tomcat
Dec 28 11:00:48 media systemd[1]: solr.service: Control process exited, code=exited status=1
Dec 28 11:00:48 media systemd[1]: solr.service: Failed with result 'exit-code'.
Dec 28 11:00:48 media systemd[1]: Failed to start LSB: Controls Apache Solr as a Service.
root@media:/build#

[Chris Clawson]

I guess things go 'wrong' when trying to setup solr as a service to run at system boot time. The script, 'solr.in.sh' gets written into the /etc/default  directory. It get the user/group settings of root:tomcat .

Yes, I am logged in via SSH as root. I believe the /var/solr directories are owned by tomcat:tomcat and the /var/solr/logs directory is empty. The installation directory at /opt/solr-8.11.1 and the sym link 'solr' are root:root as well as all the recursive contents. Is this right?

[Mohammad S. AlMutairi]

Honestly I'm guessing here because the lack of information about your server and or what has been done to it :-). As a first guess do you have SELinux or AppArmor installed and enabled on your server? Can you check it by typing as root the commands you see below.

# To check SELinux

sestatus

# To check AppArmor

aa-status

# I want you to send the result of this command too.

journalctl -xe

I'll walk you through it if you provide enough information to pin point the issue with your server and it's setup.  You should've installed Ubuntu 20.04 LTS instead of 18.04 LTS . See why you should've done that here https://wiki.ubuntu.com/Releases

[Chris Clawson]

This is a KVM cloud server hosted at http://www.tektonic.net/. It is a basic LAMP installation and has a Wordpress site installed (meloware.com) . I am trying to install DSpace 7 in preparation for upgrading a live database DSpace 6.3 installation on a different cloud VPS. This Ubuntu 18 vps is a service I have been using for a few years. It is not a new installation. The vps is installed in a very minimal configuration and is not likely to have any packages installed that I didn't do myself. The service allows 2 cpu cores and 4GB of ram. I have full root access and can only re-install everything if I break it. I believe Ubuntu 18 is compatible and I think I have  installed all the packages required for DSpace 7. When building this DSpace with yarn, my system ran out of memory. I was eventually able to get it to complete by shutting down Tomcat during the build process.

The command 'sestatus' was not available as a command, so I installed policycoreutils. Now the command says "SELinux status:                 disabled".

The command, aa-status, produced the following:

root@media:/# aa-status
apparmor module is loaded.
10 profiles are loaded.
10 profiles are in enforce mode.
   /sbin/dhclient
   /usr/bin/man
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/NetworkManager/nm-dhcp-helper
   /usr/lib/connman/scripts/dhclient-script
   /usr/sbin/mysqld
   /usr/sbin/ntpd
   /usr/sbin/tcpdump
   man_filter
   man_groff
0 profiles are in complain mode.
1 processes have profiles defined.
1 processes are in enforce mode.
   /usr/sbin/mysqld (867)
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.
root@media:/#

It looks like someone is hammering ports for my root access. This  IP  221.131.165.50 is not anything I am part of and is probably a hacker. Here are the last few lines from the journal:

Dec 28 12:15:11 media sshd[2824]: Disconnected from authenticating user root 221.131.165.50 port 19567 [preauth]
Dec 28 12:15:11 media sshd[2824]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=221.131.165.50  user=root
Dec 28 12:15:13 media sshd[2826]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=221.131.165.50  user=root
Dec 28 12:15:15 media sshd[2826]: Failed password for root from 221.131.165.50 port 16020 ssh2
Dec 28 12:15:17 media sshd[2826]: Failed password for root from 221.131.165.50 port 16020 ssh2

*****

Isn't the problem now related to permissions and setting up solr as a startup service? I can always change any user:group ownership as needed. When I used the DSpace 7 installation page, Solr would only install without making any changes to the owners or permissions. Solr only installed when the default 'solr' user was created. Any attempt to mention 'tomcat' resulted in the same error I am seeing now, when it seems the solr.service is being setup.

I appreciate this help! DSpace is far more valuable than simply confining it to universities. There are many civil organizations in the world, which have major private collections and need to share them. Besides, many of we historians are dying off from old age. If we can't organize these collections and contribute our historic metadata, what happens to the history after we are all gone?

[Mohammad S. AlMutairi]

I was able to replicate the issue you have (see the attached snapshot). It turned out it's happening when the user tomcat is defaulted to have the login shell in /etc/passwd set to /sbin/nologin .. To resolve it you need to execute the commands you see below in the sequence you see them and then start the solr installation in the first post.

1) usermod -d /home/tomcat -s /bin/bash tomcat
2) mkhomedir_helper tomcat
3) passwd tomcat

Good luck

[Mohammad S. AlMutairi]

Is it possible to to send me the result of this command?. 

grep tomcat /etc/passwd

[Chris Clawson]

I will try that. I need to give it a rest and regroup my thinking. After I have summarized what I have done, I can ask more intelligent questions, if needed. I am guessing it is late for you, so thanks and we may talk later.

C.

[Chris Clawson]

Oops! I just saw this question after making the changes to the tomcat user. The command now produces:

$ grep tomcat /etc/passwd
tomcat:x:1003:1004::/home/tomcat:/bin/bash

[Mohammad S. AlMutairi]

It is a little bit late here. Looking at what you posted as a tomcat entry in the /etc/passwd file showed the tomcat user was either created manually or by the solr installation script but was not created by apt ( apt install tomcat9 ) so here comes a couple of questions. How did you install tomcat? and which service did you install first tomcat? or solr?. See how the entry looks like if it was installed by apt. ( tomcat:x:999:999:Apache Tomcat:/:/sbin/nologin ). What I mean Apache Tomcat is what's in the name field.

[Chris Clawson]

A bit late?? I will have this answered tomorrow (12/28). Solr now installed completely and I have fixed other errors. My web browser shows loads of CORS errors but less than I had.

I will make a summary and answer the tomcat install question later.

My Rest runs out of tomcat. You can view quite a summary by looking at my HAL browser at: https://meloware.com:8443/#/api  SSL port 8443 is where my DSpace 6.3 install runs at https://montaguearchive.org:8443/ . Maybe I have a problem using this port. Thanks and good night.

[Mohammad S. AlMutairi]

You are most welcome. The CORS errors are fixable. See if you can get anything out of what I'm trying to convey to fix it.

I'm assuming you have dspace installed into /opt/dspace and also the installed tomcat version is ver9 and tomcat was installed using ubuntu advanced package tool (apt) and the dspace designated server name is meloware.com and ports 80,443 (http/https) are available to be used for meloware.com.

################################################ Tomcat Checking ########################################################
# You need to replace Catalina connector element in /etc/tomcat9/server.xml with what you see below in red.
vi /etc/tomcat9/server.xml

    <Connector address="127.0.0.1" port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443"
               maxHttpHeaderSize="65536"
               minSpareThreads="25"
               enableLookups="false"
               disableUploadTimeout="true"
               URIEncoding="UTF-8"/>


# You also need to tell tomcat it can read & write in DSpace installation folder /opt/dspace/. The ReadWritePaths line you see below should be added after/below? the ReadWritePaths=/var/lib/tomcat9/webapps/ line.
vi /lib/systemd/system/tomcat9.service
ReadWritePaths=/opt/dspace/

# You need to tell systemd manager configuration to scan all units for changes and reload them & you also need to restart the tomcat service for the changes to take effect.
systemctl daemon-reload
systemctl restart tomcat9

#  Make sure for the last time permissions are set correctly for the user tomcat on dspace installation folder.
chown -R tomcat:tomcat /opt/dspace

########################################### End of Tomcat Checking ########################################################



##################################### dspace main config files Checking ###################################################
# I'm also assuming with this configuration backend and frontend servers resides on the same host server ( same web server) and do have the same domain name.
# In local.cfg or dspace.cfg whichever cfg file you change you need to double check the servers names.

dspace.server.url = https://meloware.com/server  <--- should be set to the public api server url
dspace.ui.url = https://meloware.com             <--- Should be set to the public frontend url
solr.server = http://localhost:8983/solr         <--- Should be left as is and should never be made publicly accessible. It's better if you bind it to the loopback adapter (127.0.0.1) to be safer.


# In the environment.prod.ts config file you need to double check the UI and the REST portions as you can see below.
  ui: {
    ssl: false,                  <--- Leave it as it is.
    host: 'localhost',       <--- Leave it as it is.
    port: 4000,               <--- Leave it as it is.

  rest: {
    ssl: true,               <--- Leave it as it is. Apache should be already configured for this to be true so keep it true and make sure you can access the backend server https://meloware.com/server and it's working before doing the next step.
    host: 'meloware.com',    <--- change it to the server name you picked in local.cfg/dspace.cfg as a value for dspace.server.url.
    port: 443,
    nameSpace: '/server',    <--- Leave it as it is.

# You need to check
yarn config:check:rest

RESPONSE: 200     <--------- you should get the 200 response for a successful connection test.

Checking JSON returned for validity...
        "dspaceVersion" = DSpace 7.1.1
        "dspaceUI" = https://meloware.com      <--- should match dspace.ui.url in local.cfg or dspace.cfg
        "dspaceServer" = https://meloware.com/server    <--- should match dspace.server.url in local.cfg or dspace.cfg
        "dspaceServer" property matches UI's "rest" config? true      <--- Should be always true before you proceed to the building step
        Does "/api" endpoint have HAL links ("_links" section)? true  <--- Should be always true before you proceed to the building step


#################################  End of dspace main config files Checking ###############################################




#################################  Apache virtual host & Reverse proxy Configuration ###############################################

# You need to enable these modules mainly the proxy_http for the reverse proxy to work.
a2enmod proxy_http ssl headers alias include

# You need to create an apache virtual hosting config file for dspace. Copy what's between the red lines below and paste it into dspace.conf .

# You must change the certificates/key names and change the paths for these files..

vi /etc/apache2/sites-available/dspace.conf

--------------------------------------------------------------------------------------------------------------------------------------------------
<VirtualHost *:80>
     ServerName meloware.com
     Redirect / https://meloware.com/
</VirtualHost>

<VirtualHost *:443>
     ServerName meloware.com
     LogLevel warn
     ErrorLog ${APACHE_LOG_DIR}/meloware.com.error.log
     CustomLog ${APACHE_LOG_DIR}/meloware.com.access.log combined
     ProxyRequests On
     SSLEngine on
     SSLProxyEngine on
     SSLCertificateFile /etc/ssl/certs/CHANGE _THIS_.crt
     SSLCertificateKeyFile /etc/ssl/private/CHANGE_THIS.key
     SSLCertificateChainFile /etc/ssl/certs/SectigoCA.crt
     SSLCompression off
     ProxyPreserveHost on
     SSLProtocol             all -SSLv3 -TLSv1 -TLSv1.1
     SSLCipherSuite          ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
     SSLHonorCipherOrder     off
     SSLSessionTickets       off
     RequestHeader set X-Forwarded-Proto "https" env=HTTPS

   <Proxy *>
      AddDefaultCharset Off
      Order allow,deny
      Allow from all
   </Proxy>

     ProxyPass /server http://localhost:8080/server
     ProxyPassReverse /server http://localhost:8080/server
     #ProxyPass /solr http://localhost:8983/
     #ProxyPassReverse /solr http://localhost:8983/
     ProxyPass / http://localhost:4000/
     ProxyPassReverse / http://localhost:4000/
</VirtualHost>
--------------------------------------------------------------------------------------------------------------------------------------------------

# Enable the dspace site you just created in the step above.

a2ensite dspace.conf

systemctl restart apache2
#################################  End of Apache virtual host & Reverse proxy Configuration ###############################################

Finally, I hope I was able to help you or at least was able to clear where you might have gone wrong during the configuration. If there is anyone to thank it should go to the beautiful minds behind DSpace. Who consumed their time and energy to make it what it is today. Thank you and you have our respects.

--
All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx
---
You received this message because you are subscribed to a topic in the Google Groups "DSpace Technical Support" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dspace-tech/vgGzkRSENtk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dspace-tech...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/d0a50b87-0665-4653-9674-190afd01939fn%40googlegroups.com.

[Chris Clawson]

Please allow a bit of 'off topic' comment, so you and the DSpace documentation folks may appreciate what I am doing. I had been out of school more than ten years before the first 8 bit PCs became available in the home. I have no direct contact with any I.T. Professionals and needed to learn everything I know off of the internet. All of my cyberspace learning has been self directed and focused specifically on reaching my project goals, rather than getting a broad education on any topic. DSpace is a valuable tool I am trying to promote for community historical societies and small museums.

I am fairly familiar with a LAMP server. I understand how to install and configure a website. My live site is running both Wordpress (with LAMP) and DSpace 6.3 (Tomcat). This Tomcat has been my first exposure with the idea of a java server (or java anything).

Respectfully, these installation instructions for DSpace are the best I have available, but often leave me guessing about details in procedure or direct me elsewhere for further instruction. I am often confused at those sites or find that the topic has nothing to do with installing a module in a remote web server. Mohammad is asking me now about how and why I did things in a certain way. The general answer is that I was unable to see a clear procedure defined in the DSpace installation page and needed to read and try stuff elsewhere, in order to be able to continue my project. This help you all are offering is leading me through a 'dry run' before I attempt to upgrade my live server at montaguearchive.org . If I can learn this properly, I might consider working on a tutorial video and documentation on installing DSpace (with the blessing and guidance of the folks at Lyrasis). If I can understand this, at my level of skill, I may be able to present it in a way that could be appreciated by a much wider audience of folks.

*******

Mohammad:

The DSpace installation page didn't seem to provide any specific procedure for installing Tomcat. I mostly followed the tutorial presented at https://www.hostinger.com/tutorials/how-to-install-tomcat-on-ubuntu/ . I had installed Tomcat before I attempted an install of Solr. I do not remember ever defining a user or password for the user/group tomcat. The usermod, mkhomedir_helper, and passwd setup was critical in getting the install_solr_service.sh script to run! I would never have figured this out on my own and I thank you again!

I re-ran the following commands:

curl http://localhost:8983/solr/admin/cores
curl http://localhost:8983/solr/admin/cores?action=STATUS

There were a number of permission errors. Here was the condition of a directory before I chown'd everything to tomcat:tomcat :

root@media:/var/solr/data#

drwxr-xr-x 3 root root 4096 Dec 28 17:25 authority

drwxrwxr-x 2 tomcat tomcat 4096 Dec 28 17:25 filestore

drwxr-xr-x 3 root root 4096 Dec 28 17:25 oai

drwxr-xr-x 3 root root 4096 Dec 28 17:25 search

-rw-r----- 1 tomcat tomcat 2427 Dec 28 17:22 solr.xml

drwxr-xr-x 3 root root 4096 Dec 28 17:25 statistics

drwxrwxr-x 2 tomcat tomcat 4096 Dec 28 17:25 userfiles

-rw-r----- 1 tomcat tomcat 1048 Dec 28 17:22 zoo.cfg

root@media:/var/solr/data#

The curl status commands are reporting a lot of detail about Solr and more than I should paste into this post.

PM2 is installed, but I need to start it manually. How can I make it run when the system boots?

I just got up and see that you have sent me a lot of information, related to the status you think my install is at this morning. Thanks and give me time to study it all!

C.

[Chris Clawson]

This current effort is just practice for bigger things. Perhaps I should backup this work and then begin again with a fresh install. Many questions have been answered here and a second try should go much faster. My installation still has differences with the documentation, which may be fixable, but could still complicate any support I might hope to get later on.

A second try will also give me the opportunity to create and verify a checklist procedure for future installations. This might be useful to others later on.

Mohammad, you have been a great amount of help. I also offer my respects and appreciation to everyone involved with DSpace development. This is a valuable project and I hope my experience with it will make a small contribution to it's acceptance to the wider global community.

Chris Clawson

[Mohammad S. AlMutairi]

On Wednesday, December 29, 2021 at 4:03:40 PM UTC+3 Chris Clawson wrote:

Mohammad:

The DSpace installation page didn't seem to provide any specific procedure for installing Tomcat.

Hello Chris,

Not a specific step by step to follow for first timers but if you re-read the tomcat installation instructions and the other installations you will see almost everything an installer might need to know is pointed at or hinted. https://bit.ly/32DqZvw

 

I mostly followed the tutorial presented at https://www.hostinger.com/tutorials/how-to-install-tomcat-on-ubuntu/ .

That's an outdated article. The author in the summary of that article says Tomcat6 is the only tomcat package available to install on Ubuntu 18.04. Nowadays Tomcat8 & Tomcat9 are available on Ubuntu 18.04 you can check them out if you update and upgrade your server as below. Using package managers (apt) to install packages will take care of future updates, bug fixes and security patches for.

a) apt update -y

b) apt upgrade -y

c) apt-cache search tomcat | grep "^tomcat"

I had installed Tomcat before I attempted an install of Solr. I do not remember ever defining a user or password for the user/group tomcat. The usermod, mkhomedir_helper, and passwd setup was critical in getting the install_solr_service.sh script to run! I would never have figured this out on my own and I thank you again!

Now we know what was the screaming all about :-). There are missing steps in the Solr installation instructions I posted. If you please can help in re-writing these on the installation document you working on to make it easier for the first timers.

# Changing the home dir and login shell for tomcat won't work if tomcat is running and enabled so you need to stop it and disabled it temporarily until you finish solr installation.

1) mkdir /build
2) cd /build
3) wget https://downloads.apache.org/lucene/solr/8.11.1/solr-8.11.1.tgz
4) tar xzf solr-8.11.1.tgz solr-8.11.1/bin/install_solr_service.sh --strip-components=2
5) perl -i -pe 's/SOLR_USER=solr/SOLR_USER=tomcat/;' /build/install_solr_service.sh

6) systemctl stop tomcat9.service

7) systemctl disable tomcat9
8) usermod -d /home/tomcat -s /bin/bash tomcat
9) mkhomedir_helper tomcat
10) passwd tomcat
11) bash ./install_solr_service.sh solr-8.11.1.tgz -f
12) echo SOLR_OPTS=\"\$SOLR_OPTS -Dsolr.allowPaths=/opt/dspace/solr/statistics,/opt/dspace/temp/solr-data\" >> /etc/default/solr.in.sh
13) cp -r /opt/dspace/solr/* /var/solr/data/ # Do this step after installing dspace backend (REST API server). You need to change /opt/dspace to the folder you installed dspace backend into.
14) chown -R tomcat:tomcat /opt/sol*
15) chown -R tomcat:tomcat /var/solr/data/
16) systemctl enable solr

17) systemctl restart solr
18) systemctl enable tomcat9
19) systemctl start tomcat9.service

I re-ran the following commands:

curl http://localhost:8983/solr/admin/cores
curl http://localhost:8983/solr/admin/cores?action=STATUS

There were a number of permission errors. Here was the condition of a directory before I chown'd everything to tomcat:tomcat :

Please do this:

1) grep RUNAS= /etc/init.d/solr

2) chown -R tomcat:tomcat /opt/dspace

3) chown -R tomcat:tomcat /opt/solr*

4) chown -R tomcat:tomcat /var/solr/

After setting a login shell for the user tomcat you should always login with the user tomcat to manage and administer dspace and solr because if you use root to manage dspace  it's possible you might leave back files or directoies owned by root in the dspace installation folder which will create a permission error for the tomcat user to change them, delete them, move them ..etc.

PM2 is installed, but I need to start it manually. How can I make it run when the system boots?

As I told you before I used to use pm2 to start the frontend but I'm working on something else which I will post here once I'm finish testing it for you to include it in your doc.

############### run the service using pm2   ###############
# Create the service startup script (dspace-angular.json).
vi /opt/dspace-angular/dspace-angular.json
{
    "apps": [
        {
            "name": "dspace-angular",
            "cwd": "/opt/dspace-angular",
            "script": "yarn",
            "args": "run serve:ssr",
           "interpreter": "none"
        }
    ]
}

pm2 startup # <-- Should create a systemd startup script
pm2 start /opt/dspace-angular/dspace-angular.json
pm2 save
pm2 ls or pm2 status # To check the status

systemctl enable pm2-root.service
systemctl restart pm2-root.service

systemctl status pm2-root.service
###########################################################

I just got up and see that you have sent me a lot of information, related to the status you think my install is at this morning. Thanks and give me time to study it all!

C.

Take your time to digest it. All those details are summarized from DSpace official site or this group list.

[Mohammad S. AlMutairi]

Chris:

I never asked you why you are using port 8443 in your old server and the new one. Not sure why but in case you must use it and you must run DSpace on it the steps below is what you need to make it work Apache and DSpace work. One note you must pay attention two. You must use the file I sent you or at least use a reverse proxy in your apache configuration file to talk to tomecat and the frontend on port 8080 and 4000.

First step you need to make sure apache are configured for that port in two files:

vi /etc/apache2/ports.conf

# -----------------------------------------------------------------------------------

Listen 80

<IfModule ssl_module>
        Listen 443
        Listen 8443
</IfModule>

<IfModule mod_gnutls.c>
        Listen 443
        Listen 8443
</IfModule>

# -----------------------------------------------------------------------------------

Also you need to change meloware apache virtual host config file that I posted in the group list to run on that port:

vi /etc/apache2/sites-available/meloware.conf

<VirtualHost *:8443>

# Restart Apache

systemctl restart apache2

What you need then is to change local.cfg or dspace.cfg

dspace.server.url = https://meloware.com:8443/server

https:// meloware.com:8443

# In environment.prod.ts change the port for the front rest server only. Just leave the UI portion to localhost and port 4000

  rest: {
    ssl: true,
    host: ' meloware.com ',
    port:  8443,
    nameSpace: '/server',

# Check to see if everything is fine

yarn config:check:rest

# Build the frontend and restart it once everything is right.

yarn run build:prod

Hope it help.

To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/b506b622-207e-46a4-8f49-39ada4d7b2cfn%40googlegroups.com.

[Chris Clawson]

Thanks -

I am in the process now of reviewing everything I have learned and you have written out. I am attempting a re-install of Tomcat, Solr and will pay close attention to all the Apache proxies and other configs you have defined in this thread. PostgreSQL was a pain, but I think I have it right. I will leave that and my other backend prerequisites  as they are now. So my plan is to get the backend running again and then proceed following your directions. If I screw up, I will just restore my backup image and start over.

montaguearchive.org is over a year old and I am a bad book keeper. The port 8080 is to access the Wordpress html/php site and 8443 is to reach Tomcat and DSpace 6.3. I think I realize this method is probably going to change with DSpace 7. For now, I will just break the wordpress install of meloware and focus on getting a functional DSpace server running with some of the montaguearchive data. My partner is totally non technical (and even older than I am), so he is going to need to be persuaded that upgrading his project is a good idea. (So am I). I hope DSpace 7 has some great and modern media handling features I can brag about. I also want to try out the Jpeg2000 support. Cloud storage is expensive and some of my great Seadragon image pyramids are over 2 GB in size! It would be great to shrink that.

I will study this last post carefully, once I get to setting up Apache.

[Chris Clawson]

Status and Tomcat configuration questions...

I have wiped my server of any previous installations of Solr and Tomcat (I think). I have been reviewing this thread and writing my out my procedure (to share later), but need to leave the work for New Years day now. Solr was installed first and then Tomcat 9 afterwards.

Any effort I made to install Tomcat 9 using 'apt-get install tomcat9' did not install a binary build of apache-tomcat-9.0.56 (I don't know why). I have been following what seems to be a good manual procedure at https://www.makeuseof.com/a-step-by-step-guide-to-installing-apache-tomcat-linux/ . Both Solr and Tomcat9 now seem to be running well as services.

I am leaving the work day with the problem of configuring the server.xml and checking Tomcat instructions you have in the post “checking tomcat” (about 6 posts earlier).

1. I do not have any directory '/etc/tomcat'. I plan on editing the server.xml from the /opt/tomcat/latest/conf directory and then copy that directory's contents and paste it to a new /etc/tomcat directory (my tomcat9 service is actually named 'tomcat.service'). Let me know if I am wrong, but I think the 'Catalina' folder and other contents should be copied there too. Is this a correct step to take?

2. I am looking at what you call the 'Catalina connector element' and am not sure where that is. I see no mention of a localhost connector at 127.0.0.1 port= 8080. Is this a totally new and added connector or a modification of an existing one? Where would it be if I was looking at the default server.xml from the distribution?

3. Doesn't this server.xml also have a edited SSL Connector for port 8443? I presume the '<Connector port="8443" protocol="org.apache.coyote.http11.Http11AprProtocol" Still needs all the keys paths to the SSL certificates. Here is the connector I have running with the montaguearchive.org system, using DSpace 6.3:

<Connector

protocol="org.apache.coyote.http11.Http11NioProtocol"

port="8443"

maxThreads="200"

scheme="https"

secure="true"

SSLEnabled="true"

keystoreFile="/opt/tomcat/apache-tomcat-8.0.37/ssl/montaguearchive_org.pfx"

keystorePass="XXXXXXXX"

keystoreType="PKCS12"

clientAuth="false"

sslProtocol="TLS"/>

I am optimistic there is enough information and help here to get this all working well. Happy New Year to all and I hope the future remains tolerant of all these lay questions!

[Mohammad S. AlMutairi]

Hi Chris,

Happy New Year to all.

On Sat, Jan 1, 2022 at 11:01 PM Chris Clawson <industr...@gmail.com> wrote:

Status and Tomcat configuration questions...

I have wiped my server of any previous installations of Solr and Tomcat (I think). I have been reviewing this thread and writing my out my procedure (to share later), but need to leave the work for New Years day now. Solr was installed first and then Tomcat 9 afterwards.

Any effort I made to install Tomcat 9 using 'apt-get install tomcat9' did not install a binary build of apache-tomcat-9.0.56 (I don't know why). I have been following what seems to be a good manual procedure at https://www.makeuseof.com/a-step-by-step-guide-to-installing-apache-tomcat-linux/ . Both Solr and Tomcat9 now seem to be running well as services.

You are supposed to be able to install tomcat9 using 'apt install tomcat9' if your server is updated using 'apt update -y' and upgraded using 'apt upgrade -y' and the default Ubuntu 18.04 repositories are configured and enabled ( I'm suspecting your server is not updated. https://bit.ly/3EOaJFb ). When you tried to install tomcat9 did apt display any error?. Bear with me I'll walk you through it with the current tomcat installation you did but I'm curious to know why you had an issue installing tomcat9 using apt and to prevent you from facing the same issue in the future when you need to install any other packages. ( Let's leave that a later time ). Can you please send me the results of these commands?.

1) lsb_release -a

2) grep -r ^deb /etc/apt/sources.list*

3) grep tomcat /etc/passwd

I am leaving the work day with the problem of configuring the server.xml and checking Tomcat instructions you have in the post “checking tomcat” (about 6 posts earlier).

Enjoy your holiday and don't think or worry about it. You are almost there.

1. I do not have any directory '/etc/tomcat'. I plan on editing the server.xml from the /opt/tomcat/latest/conf directory and then copy that directory's contents and paste it to a new /etc/tomcat directory (my tomcat9 service is actually named 'tomcat.service'). Let me know if I am wrong, but I think the 'Catalina' folder and other contents should be copied there too. Is this a correct step to take?

It's not a correct step to take. Don't create any extra tomcat folders or copy any files from /opt/tomcat folder. The /etc/tomcat folder should only be seen in /etc  folder if 'apt install tomcat9' succeeded installing tomcat9 because ( In a simple way ) that's how the Ubuntu tomcat9 package maintainer customize the tomcat9 installation script to do. For the startup script name or what's so called in systemd unit file 'tomcat.service' it's O.K and perfectly fine. It can be named anything as long as you remember the script name to handle the service 'systemctl stop|start|restart|status tomcat.service'.

1. I am looking at what you call the 'Catalina connector element' and am not sure where that is. I see no mention of a localhost connector at 127.0.0.1 port= 8080. Is this a totally new and added connector or a modification of an existing one? Where would it be if I was looking at the default server.xml from the distribution?
   

It's  a modification of an existing one. Please don't get confused here. There are two files named server.xml to work with. The one we will change the Catalina connector element in is tomcat main configuration. In your new tomcat installation it should be located here ( /opt/tomcat/latest/conf/server.xml ). The second file the ( Context file name ) which we need to create to tell tomcat where to find the dspace server webapp (/opt/dspace/webapps/server) and initialize it during tomcat start should be created here ( /opt/tomcat/latest/conf/Catalina/localhost/server.xml ). If you ask why we have to name the context file (second file) server.xml it's because tomcat dictates the context file name should have the name of the webapp name and since dspace webapp is named server the file we need to create should be named server.xml. To get a grip of what I'm trying to tell you please read DSpace Backend installation instructions ( Step 11. Deploy Server web application ) https://bit.ly/3pKbTNy

Step 1

Let's do the Catalina Connector part first. You need to edit  /opt/tomcat/latest/conf/server.xml and replace the lines numbered 69,70 and 71 that you see in blue below with the text you see in red. Please note: I've change tomcat port to be 8081 instead of 8080 because of what mentioned previously 8080 is used for Wordpress also I binded the tomcat service to listen on the loopback adapter (127.0.0.1) and so you must use a reverse proxy in the Apache web config file to do the talking to tomcat. NOTE: The tomcat service won't be accessible from outside the localhost directly so using a reverse proxy is a must for this configuration.

a) cp /opt/tomcat/latest/conf/server.xml /opt/tomcat/latest/conf/server.xml-orig

# just copy all the text you see below in red and paste it into your putty while you logged in as root and hit enter after deleting/commenting out those lines in blue.

b) vi /opt/tomcat/latest/conf/server.xml

Line 69     <Connector port="8080" protocol="HTTP/1.1"
Line 70                connectionTimeout="20000"
Line 71                redirectPort="8443" />

    <Connector address="127.0.0.1" port="8081" protocol="HTTP/1.1"
               connectionTimeout="20000"

               maxHttpHeaderSize="65536"
               minSpareThreads="25"
               enableLookups="false"
               disableUploadTimeout="true"
               URIEncoding="UTF-8"/>

 

Step 2

a) Let's create the context file for the dspace server webapp ( just copy all the text you see below in red and paste it into your putty while you logged in as root and hit enter ).

cat << EOF >> /opt/tomcat/latest/conf/Catalina/localhost/server.xml

<Context docBase="/opt/dspace/webapps/server"
         privileged="true"/>
EOF

b) This is an extra step just to make sure you didn't miss doing these steps during your installation of tomcat and java steps previously ( Just copy all the text you see below in red and paste it into your putty while logged in as root and hit enter ).

echo "export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64/
export CATALINA_HOME=/opt/tomcat/latest" >> /etc/environment

source /etc/environment

1. Doesn't this server.xml also have a edited SSL Connector for port 8443? I presume the '<Connector port="8443" protocol="org.apache.coyote.http11.Http11AprProtocol" Still needs all the keys paths to the SSL certificates. Here is the connector I have running with the montaguearchive.org system, using DSpace 6.3:

This is the file (/opt/tomcat/latest/conf/server.xml) where you can configure the SSL termination for tomcat but please don't do it that way. You still can do exactly how you did it for dspace 6.3 but it's not a proper/good way of doing it for DSpace 7. For DSpace 7 it's much better to go with the Apache SSL termination and the reverse proxy setup with this you take care of the frontend and how it's accessed externally in one config file. Just follow the next steps to run dspace in a similar way to your old setup without having to have tomcat exposed or to serve public requests or to worry about tomcat SSL configuration.

Step 3

a) You need to add the Listen directive and the new port for dspace 8443 as you see it in red.

# vi /etc/apache2/ports.conf

<IfModule ssl_module>
        Listen 443
        Listen 8443

</IfModule>

b) You need to create the apache virtual host conf file. I have amended the backend reverse proxy port to 8081. Just copy all the text in between the blue lines and past it into etc/apache2/sites-available/dspace.conf

# vi /etc/apache2/sites-available/dspace.conf

######################################################################################################################################################

<VirtualHost *:8443>

     ServerName meloware.com
     LogLevel warn
     ErrorLog ${APACHE_LOG_DIR}/meloware.com.error.log
     CustomLog ${APACHE_LOG_DIR}/meloware.com.access.log combined
     ProxyRequests On
     SSLEngine on
     SSLProxyEngine on

     SSLCertificateFile /etc/ssl/certs/CHANGE-THIS.crt
     SSLCertificateKeyFile /etc/ssl/private/CHANGE-THIS.key

     SSLCertificateChainFile /etc/ssl/certs/SectigoCA.crt
     SSLCompression off
     ProxyPreserveHost on
     SSLProtocol             all -SSLv3 -TLSv1 -TLSv1.1
     SSLCipherSuite          ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
     SSLHonorCipherOrder     off
     SSLSessionTickets       off

     ProxyPreserveHost On

     RequestHeader set X-Forwarded-Proto "https" env=HTTPS

    <Proxy *>
        AddDefaultCharset Off

        Require all granted
    </Proxy>

     ProxyPass /server http://localhost:8081/server
     ProxyPassReverse /server http://localhost:8081/server

     ProxyPass / http://localhost:4000/
     ProxyPassReverse / http://localhost:4000/
</VirtualHost>

######################################################################################################################################################

c) ufw allow 8443                 # open the apache/dspace port in the firewall if you are using ufw for firewalling your server.
d) a2ensite dspace.conf       # enable the apache virtual  host for dspace.
e) systemctl enable apache2  # enable the apache service to be started at boot time.
f) systemctl restart apache2   # start/restart the pache service

After finishing the above steps make sure to have:

in local.cfg or dspace.cfg:

dspace.server.url = https://meloware.com:8443/server

dspace.ui.url =   https://meloware.com:8443

in /opt/dspace-angular/src/environments/environment.prod.ts rest config part :

  rest: {
    ssl: true,
    host: 'https://meloware.com',

    port: 8443,
    nameSpace: '/server',

When you done building the backend and frontend I hope successfully you should access your DSpace by visiting https://meloware.com:8443

I am optimistic there is enough information and help here to get this all working well. Happy New Year to all and I hope the future remains tolerant of all these lay questions!

 You should always be optimistic Chris. Please rest assure I'll help in anyway I can to get it up and running & please accept my apology. I didn't know it's you or someone from your generation at that time. I thought I'm helping a rude person who lacks the manner because of this (I will end this thread and start a new one later). I want you to accept my apology and to let me know if you need any further help in finish what you started. Happy New Year.

To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/98ac55c2-5b83-4055-a123-186568dbbe3dn%40googlegroups.com.

[Mohammad S. AlMutairi]

Please ignore line in red. It shouldn't be there. It's a mistake during the text arrangement I did.

Step 1

# just copy all the text you see below in red and paste it into your putty while you logged in as root and hit enter after deleting/commenting out those lines in blue.

b) vi /opt/tomcat/latest/conf/server.xml

[Chris Clawson]

Great.

I believe this solr and tomcat manual installation could lead to a functioning install, but would like to make a procedure which is as close as possible to your suggestions and the existing documentations. The results of the system commands you requested is attached as 'command-session.txt' in this post. I am going to hold off on doing anything further, until I see your comments.

I am thinking it would be wise to do a removal of the manually installed Tomcat 9 installation. I would appreciate a checklist of steps required, if you agree. I wonder if my failure to use apt is due to some remaining files which were not removed from a previous install. I really would like to know my problem with my failure using 'apt-get install tomcat9'.

Thanks for the patience. At the very least, my understanding of all this is becoming much more solid than it has been.

[Mohammad S. AlMutairi]

First is first. Your server is upgraded to the latest subversion available for Ubuntu 18.04 LTS (Ubuntu 18.04.6 LTS). I think tomcat9 refused to install because you installed solr first and to be specific what caused tomcat9 refused to install is the creation of the tomcat user which you did during the solr installation so to rectify from this and install tomcat9 using apt you have to follow the a --> g steps you see below by the sequence you see them:

Before running the rectification process I want you to run this command ( apt-cache search tomcat | grep "^tomcat9" ). The result should be the names of tomcat9/related packages available to be installed by apt. If you see that proceed to execute the steps as in the commands below
tomcat9 - Apache Tomcat 9 - Servlet and JSP engine
tomcat9-admin - Apache Tomcat 9 - Servlet and JSP engine -- admin web applications
tomcat9-common - Apache Tomcat 9 - Servlet and JSP engine -- common files
tomcat9-docs - Apache Tomcat 9 - Servlet and JSP engine -- documentation
tomcat9-examples - Apache Tomcat 9 - Servlet and JSP engine -- example web applications
tomcat9-user - Apache Tomcat 9 - Servlet and JSP engine -- tools to create user instances

a) systemctl disable tomcat.service

b) systemctl stop tomcat.service

c) mv /etc/systemd/system/tomcat.service /opt/tomcat/tomcat.service

d) systemctl disable solr

e) systemctl stop solr

f) deluser --remove-home tomcat

g) apt install tomcat9

After tomcat installed successfully just follow the installation of solr you previously followed. Let me know if you need any help. I'm feeling guilt :-).

[Chris Clawson]

Okay, my Friend

I hope you are enjoying this and it is not getting annoying..

Yes, everything you mentioned about tomcat9 listed as you have shown. I then followed steps 'a' thru 'g'. (Can these services simply be disabled and then re-installed, without actually purging the old files out of the system?)

The 'apt install tomcat9' command said it intended to install 14Mb of new files.. I said Yes(of course).

I am now able to find new directories and files:

/usr/share/tomcat9
/etc/tomcat9
/var/lib/tomcat9
/etc/systemd/system/multi-user.target.wants/tomcat9.service

# I ran the following and had these responses about 'tomcat'
systemctl --no-pager
tomcat.service                               not-found failed failed    tomcat.service
tomcat9.service                              loaded    failed failed    Apache Tomcat 9 Web Appli

# Here is the status report on the tomcat9.service
systemd status tomcat9
Jan 02 12:01:15 media systemd[1]: Starting Apache Tomcat 9 Web Application Server...
Jan 02 12:01:15 media systemd[1]: Started Apache Tomcat 9 Web Application Server.
Jan 02 12:01:15 media systemd[40090]: tomcat9.service: Failed to apply ambient capabilities (before UID change): Invalid argument
Jan 02 12:01:15 media systemd[40090]: tomcat9.service: Failed at step CAPABILITIES spawning /bin/sh: Invalid argument
Jan 02 12:01:15 media systemd[1]: tomcat9.service: Main process exited, code=exited, status=218/CAPABILITIES
Jan 02 12:01:15 media systemd[1]: tomcat9.service: Failed with result 'exit-code'.

#from /etc/passwd the only user named anything, like 'tomcat' is

tomcat:x:999:999:Apache Tomcat:/:/sbin/nologin

I have not re-booted the machine, since this install attempt. Is there a problem here or a missed step?

[Mohammad S. AlMutairi]

On Sun, Jan 2, 2022 at 9:31 PM Chris Clawson <industr...@gmail.com> wrote:

Okay, my Friend 

I hope you are enjoying this and it is not getting annoying..

 Not exactly but because of you it's enjoyable for an old timer too. Don't worry about me. 

Yes, everything you mentioned about tomcat9 listed as you have shown. I then followed steps 'a' thru 'g'. (Can these services simply be disabled and then re-installed, without actually purging the old files out of the system?)

This is a new server. Right? Nothing much in the installation for both solr and tomcat you should worry about. What I did was I just disabled the old tomcat service and moved it's startup file back to /opt/tomcat folder for a later removal so it's there intact. For solr if you followed the last updated solr installation procedure where the forcing option is used it should've overwritten the old solr directories but kept the dspace cores intact if you already copied them at the time you re-install solr. Anyway, the log you sent doesn't say much but the exit status was enough to find what could the issue you have starting tomcat. Can you check the kernel version with this command ( uname -r ) and send it over?. Try doing the step below:

a) vi /lib/systemd/system/tomcat9.service

# You need to comment out that line that says AmbientCapabilities=CAP_NET_BIND_SERVICE

 30 User=tomcat
 31 Group=tomcat
 32 PrivateTmp=yes

 33 #AmbientCapabilities=CAP_NET_BIND_SERVICE

b) systemctl daemon-reload

c) systemctl restart tomcat9.service

d) systemctl status tomcat9.service

Let me know if you need any more help. I'll be very glad to help in any way.

To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/ba58dced-6fe9-4cab-ae52-3605b1dc1930n%40googlegroups.com.

[Chris Clawson]

Tomcat9 runs!

This is something I would never have discovered and proves I need this help. Your answer to the uname command is here:

root@media:~# uname -r
3.13.0-45-generic

No, this is not a new server. It has been online a while and may have many things wrong with it. The LAMP has been running for over a year. I may have made other changes. Maybe I am over the problems and can finish this without much further help. Maybe not. I have also just started a home network fresh build of a PC with Ubuntu 20.04.3 and will see if I can do a DSpace7.1 install on that. That procedure should be the same, until the SSL stuff is encountered.

I am the 'chris' an the contact page at montaguearchive.org . I don't know if this conversation is proper to continue forever in this forum and you can always reach me with the email address at that page. I will take some time now to study the recent posts and see if I can make a coherent procedure out of it.

[Mohammad S. AlMutairi]

On Monday, January 3, 2022 at 1:40:26 AM UTC+3 Chris Clawson wrote:

Tomcat9 runs!

Glad it's sorted out.

 

This is something I would never have discovered and proves I need this help. Your answer to the uname command is here:

root@media:~# uname -r
3.13.0-45-generic

It's an old version.  kernel version 4.15.0-163-generic is the latest for Ubuntu 18.04 LTS. ( Is the kernel updates disabled in your server? ) You should think of updating the kernel and it's better if your ask tektonic support to do it for you because it might fail to boot after the kernel updates. I've seen it happening before.

 

No, this is not a new server. It has been online a while and may have many things wrong with it. The LAMP has been running for over a year. I may have made other changes. Maybe I am over the problems and can finish this without much further help. Maybe not. I have also just started a home network fresh build of a PC with Ubuntu 20.04.3 and will see if I can do a DSpace7.1 install on that.

Think of doing it on a Hyper-V for testing & development purposes. It's free and most latest Windows versions have it built-in and ready to be installed.

 

That procedure should be the same, until the SSL stuff is encountered.

Same procedure and steps should be followed on both Ubuntu versions if that's what you mean with the exception PostgreSQL v12 should be installed using PostgreSQL official repository for Debian/Ubuntu in Ubuntu 18.04 LTS.

 

I am the 'chris' an the contact page at montaguearchive.org . I don't know if this conversation is proper to continue forever in this forum and you can always reach me with the email address at that page. I will take some time now to study the recent posts and see if I can make a coherent procedure out of it.

You have my email. Don't hesitate to ask for help to finish your installation & try it.

 

Good luck

[Chris Clawson]

I have normalized my DSpace 7 install as closely as I am able, and try to match it with the instruction page and the advice from this thread. The backend seems okay and may be viewed at the appropriate location for meloware.com. I am sorry about the confusion over the use of ports 443 and 8443. This confusion may still be an issue for me getting this right.

Solr and Tomcat are running well. The issue with apt-get install tomcat9 may be that it did try to install, but failed because it wanted the tomcat user as a user configured with a home directory and login. I still had to do this to get tomcat9 to install.

With this site 'dspace.conf',  Apache won't start if line 49 is enabled. The syntax seems correct and I would welcome a review of it. Apache won't start at all if this “RequestHeader set X-Forwarded-Proto "https" env=HTTPS” line is in the virtual host configuration.

There probably are other problems. I am getting “mixed content” errors (in my Firefox console) and it may be because of this file.

If I allow insecure content in my browser, the search and browse buttons know the database is empty.

Okay. Things aren't stable and the config files may not be totally right.

Attached are my tomcat9's server.xml and the Apache site config file dspace.conf

[Mohammad S. AlMutairi]

Hi Chris,

Can you replace line 49 with RequestHeader set X-Forwarded-Proto "https" and then run the commands you see below and send me the apache version command output.

a) apachectl configtest

b) apachectl -v

c) systemctl restart apache2

d) systemctl status apache2

Is this the old server?. I think I can help save your time so how what you think inviting me using TeamViewer to help you have this server setup done for you?. 

[Chris Clawson]

This is the same server we have been troubleshooting in the thread. I haven't used Teamviewer but am very open to direct help if you are offering. My goal is to understand this process so I can use it elsewhere. This server is my education, and really anything can be done with it. I have a client who uses FTP but that isn't hard to keep. I need your direct contact. I have no email for you, but you can reach me on the montaguearchive.org contact page.

Replacing the line 49 produces the same error I have been seeing. Below is the terminal messages from your instructions after editing the file.

root@media:/dspace-tmp# apachectl configtest
AH00526: Syntax error on line 49 of /etc/apache2/sites-enabled/dspace.conf:
Invalid command 'RequestHeader', perhaps misspelled or defined by a module not included in the server configuration
Action 'configtest' failed.
The Apache error log may have more information.
root@media:/dspace-tmp# apachectl -v
Server version: Apache/2.4.29 (Ubuntu)
Server built:   2021-11-14T23:52:18
root@media:/dspace-tmp# systemctl restart apache2
Job for apache2.service failed because the control process exited with error code.
See "systemctl status apache2.service" and "journalctl -xe" for details.
root@media:/dspace-tmp# systemctl status apache2
? apache2.service - The Apache HTTP Server
   Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
  Drop-In: /lib/systemd/system/apache2.service.d
           +-apache2-systemd.conf
   Active: failed (Result: exit-code) since Mon 2022-01-03 19:21:30 CST; 18s ago
  Process: 3901 ExecStop=/usr/sbin/apachectl stop (code=exited, status=1/FAILURE)
  Process: 3906 ExecStart=/usr/sbin/apachectl start (code=exited, status=1/FAILURE)
 Main PID: 865 (code=exited, status=0/SUCCESS)

Jan 03 19:21:30 media systemd[1]: apache2.service: Failed with result 'exit-code'.
Jan 03 19:21:30 media systemd[1]: Stopped The Apache HTTP Server.
Jan 03 19:21:30 media systemd[1]: Starting The Apache HTTP Server...
Jan 03 19:21:30 media apachectl[3906]: AH00526: Syntax error on line 49 of /etc/apache2/sites-enable
Jan 03 19:21:30 media apachectl[3906]: Invalid command 'RequestHeader', perhaps misspelled or define
Jan 03 19:21:30 media apachectl[3906]: Action 'start' failed.
Jan 03 19:21:30 media apachectl[3906]: The Apache error log may have more information.
Jan 03 19:21:30 media systemd[1]: apache2.service: Control process exited, code=exited status=1
Jan 03 19:21:30 media systemd[1]: apache2.service: Failed with result 'exit-code'.
Jan 03 19:21:30 media systemd[1]: Failed to start The Apache HTTP Server.

[Mohammad S. AlMutairi]

I can see the cause of it I think now. You need to enable the headers module in apache. Please execute the commands you see below:

a) a2enmod proxy_http ssl headers alias include

b) apachectl configtest

c) systemctl restart apache2

d) systemctl status apache2

It should have this issue sorted out I hope.

It's getting late here I need to rest a little bit. I'll contact you by email in the early morning US time to arrange the login session. Talk to you then. Have a good night

[Chris Clawson]

That helped - a lot!

I was just able to login as DSpace administrator for the first time. I kept playing and just finished adding a Community, Collection and the first item.

I will be available in about 13 hours from this post and maybe we can talk then. Thanks so much.

Attached  is the result of these last commands.