DSpace 7.1. front page error

[Jaware]

Dear All, I'm trying local installation of DSpace 7.1 on Ubuntu 20.04 server over my laptop.
 Below is the output while making yarn start.
 
     $ node dist/server
     [HPM] Proxy created: /  -> http://localhost:8080/server/sitemaps
     [14:05:33 GMT+0300 (East Africa Time)] Listening at http://localhost:4000/
 
 I can access both the backend(http://localhost:8080/server) and front end (http://localhost:4000) smoothly.
 How ever I was looking 'error.top-level-communities' on the front page    
     
  So far I was doing as following actions
 
  1.Deploy the web application and set permission for tomcat user .

 - dspace@dspace:~$ cd /var/lib/tomcat9/webapps/

 - dspace@dspace:/var/lib/tomcat9/webapps$ sudo ln -s  /home/mydspace/webapps/server server
 
 - dspace@dspace:~$ sudo chown -R tomcat:tomcat /home/mydspace/
 
 
     
 2.Copy solr cores and set permission for solr user

-dspace@dspace:~$ sudo cp -r /home/mydspace/solr/* /opt/solr-8.11.1/server/solr/configsets/

-dspace@dspace:~$ sudo cp -r /home/mydspace/solr/* /opt/solr/server/solr/configsets/        

-dspace@dspace:~$ sudo chown -R solr:solr /opt/solr/server/solr/configsets/    
     
 3.  Modify the front-end file (environment.common.ts) as follow
 
   
   ui: {
    ssl: false,
    host: 'localhost',
    port: 4000,
    // NOTE: Space is capitalized because 'namespace' is a reserved string in TypeScript
    nameSpace: '/',
    // The rateLimiter settings limit each IP to a "max" of 500 requests per "windowMs" (1 minute).
    rateLimiter: {
      windowMs: 1 * 60 * 1000,   // 1 minute
      max: 500 // limit each IP to 500 requests per windowMs
    }
  },
  // The REST API server settings.
  // NOTE: these must be "synced" with the 'dspace.server.url' setting in your backend's local.cfg.
  rest: {
    ssl: false,
    host: 'localhost',
    port: 8080,
    // NOTE: Space is capitalized because 'namespace' is a reserved string in TypeScript
    nameSpace: '/server',
  },
     
     
 Question 1 should  solr be owned by tomcat user? and 

 Question 2.who owns dspace angular(front page) tomcat or solr
 
 Can some one help me please,
 
 Best,Jaware    
     
 
 
 
 

[Tim Donohue]

Hi Jaware,

In order to understand where the error is coming from, you should first follow our "Troubleshoot an error" guide for DSpace 7.  This will guide you through how to find detailed error messages in the User Interface and in the backend: https://wiki.lyrasis.org/display/DSPACE/Troubleshoot+an+error#Troubleshootanerror-DSpace7.x(orabove)

Once you have found a more detailed error message, you should check if it's one of the common installation issues listed at: https://wiki.lyrasis.org/display/DSDOC7x/Installing+DSpace#InstallingDSpace-CommonInstallationIssues   Chances are you are likely hitting one of these common issues, in which case the steps to resolve it are listed on that wiki page.

If you find an error that isn't one of those common issues, feel free to send it to this list and we can provide better advice then.

Tim

[Mohammad S. AlMutairi]

Hello everyone,

I need to confirm this to the community. Whenever this error 'error.top-level-communities' do show up on the front page ( should be added to the common installation issues wiki ) Solr is either not installed or misconfigured. To resolve it on the Ubuntu Linux servers just follow the steps you see below.

1) mkdir /build
2) cd /build
3) wget https://downloads.apache.org/lucene/solr/8.11.1/solr-8.11.1.tgz
4) tar xzf solr-8.11.1.tgz solr-8.11.1/bin/install_solr_service.sh --strip-components=2
5) perl -i -pe 's/SOLR_USER=solr/SOLR_USER=tomcat/;' /build/install_solr_service.sh

6) systemctl disable tomcat9

7)  systemctl stop tomcat9.service
8) usermod -d /home/tomcat -s /bin/bash tomcat
9) mkhomedir_helper tomcat
10) passwd tomcat
11) bash ./install_solr_service.sh solr-8.11.1.tgz -f
12) echo SOLR_OPTS=\"\$SOLR_OPTS -Dsolr.allowPaths=/opt/dspace/solr/statistics,/opt/dspace/temp/solr-data\" >> /etc/default/solr.in.sh
13) cp -r /opt/dspace/solr/* /var/solr/data/ # Do this step after installing dspace backend (REST API server). You need to change /opt/dspace to the folder you installed dspace backend into.
14) chown -R tomcat:tomcat /opt/sol*
15) chown -R tomcat:tomcat /var/solr/data/
16) systemctl enable solr

17) systemctl restart solr
18) systemctl enable tomcat9
19) systemctl start tomcat9.service

You need to make sure the tomcat user can write to DSpace installation folder. On Ubuntu it can be done by adding ReadWritePaths=/opt/dspace/ (change the folder name if it differs) in systemd tomcat startup script in the [service] section.

"Question 1 should  solr be owned by tomcat user?"

Easiest path the answer is yes.

"Question 2.who owns dspace angular(front page) tomcat or solr."

It can be any other user. www-data user is a good candidate on Ubuntu but it can be any other user you find safe to use. If you do that you must run the frontend with that user and make it the owner of the angular folder (/opt/dspace-angular).

[Jaware Technologies]

Dear AlMutairi,

Fantastic explanation.It works based on your comment

Thanks again,

Best,Jaware

--
All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx
---
You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/3e9d0d0d-f05f-454f-97e9-4986052906f1n%40googlegroups.com.

[Mohammad S. AlMutairi]

Hello Jaware,

You welcome & glad it worked for you. My comment is based on the excellent Dspace documentation led by Tim & the working force with him. The steps above is a summarized translation of Dspace documentation specific to Ubuntu Linux users. It's my little contribution to the community trying to save anyone's time.

Regards.

[Mark H. Wood]

On Fri, Feb 04, 2022 at 04:55:23AM -0800, Jaware wrote:
> Dear All, I'm trying local installation of DSpace 7.1 on Ubuntu 20.04
> server over my laptop.

...

> Question 1 should solr be owned by tomcat user? and

That is not required. Here, I created a separate user 'solr' to run
the Solr service. The Solr files should be owned by the user which
runs the service.

> Question 2.who owns dspace angular(front page) tomcat or solr

Perhaps neither. It should not matter, and I would again use a
separate user for dspace-angular. Again the files should be owned by
the user which runs the service.

The three processes (dspace-server-webapp, dspace-angular, solr)
interact only through network connections. They do not need direct
access to each others' files, and I would argue that they *should
not* have direct access to each others' files.

One does not even need to run these processes on the same server.

If DSpace cannot query Solr then the problem lies elsewhere. Solr
may not be running, or may have encountered a problem. DSpace may not
be correctly configured to contact the necessary Solr core. The log
files may contain useful clues.

--
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu

[Mohammad S. AlMutairi]

On Monday, February 7, 2022 at 4:14:26 PM UTC+3 Mark H. Wood wrote:

The three processes (dspace-server-webapp, dspace-angular, solr)
interact only through network connections. They do not need direct
access to each others' files, and I would argue that they *should
not* have direct access to each others' files.

One does not even need to run these processes on the same server.

Thanks for the rich information comment Mark. It seems you are running a different flavor of Linux not the Ubuntu. You need to try installing Dspace & Solr on a Ubuntu server to see what the users complains were all about. here is one https://groups.google.com/g/dspace-tech/c/z2N2BBBiu2E/m/dJyaLMZoCwAJ

[Mark H. Wood]

Well, Ubuntu Focal has Solr in its package repository, but it's a
decade old, so Ubuntu is one of the distro.s on which you need to do
the installation manually. That means you'll need to create a user to
run it, very like Gentoo which I use.

I called that user 'solr'. I unpacked the Solr v8 files in
/opt/solr-8, created /var/solr-8 to hold the configuration and cores,
created /etc/solr-8 to hold the solr.in.sh and log4j2.xml, created
/var/log/solr-8 to hold the logs, and edited solr.in.sh to set
LOG4J_PROPS, SOLR_PID_DIR, SOLR_HOME, and SOLR_LOGS_DIR
appropriately. I set the /var/lib/solr-8 directory and contents to be
owned by 'solr'. I created an initscript (Ubuntu will need a "system
unit" or some such) to start /opt/solr-8/bin/solr as user 'solr'
running in ${SOLR_HOME}, with SOLR_INCLUDE set to
'/etc/solr-8/solr.in.sh' and passing options '-p 8983 -s
${SOLR_HOME}'. Much of this is what one does for any daemon that
isn't in the package manager.

At that point Solr should start and run, with no cores. I created a
directory ${SOLR_HOME}/ds7 and copied the DSpace 7 empty cores to
there, setting them owned by 'solr'. (You can put the cores directly
into ${SOLR_HOME} but I like to keep related cores organized this
way.)

With reference to the dspace-tech discussion linked above: I don't
know where Solr is getting that path '/dspace/solr/statistics'. We
would need to see how Solr is configured at that site. Assuming that
DSpace is installed at '/dspace', then '/dspace/solr/*' should have
been copied into ${SOLR_HOME} and the copies set owned by the user
that runs Solr. The copies in '/dspace' are just templates, and are
not referenced by DSpace or Solr in use.

Setting up dspace-angular follows the same pattern, with details as
needed by nodejs: create a user to run node; put the package's files
in some place that makes sense to you such as '/opt/dspace-v7-fe';
create a directory to hold the logs such as '/var/log/ds7'; write a
startup file appropriate to your OS, passing environment properties
and options as needed; set file ownership appropriately. Debian
(thus, presumably, Ubuntu) packages a fairly recent 'nodejs' so
installing that is simple. (Hmmm: Focal is still packaging node v10
which is pretty old. Raspbian Bullseye packages v14. I haven't got
an Impish instance to check. Gentoo packages v14.17.6 so that's what
I'm using.)

[Mohammad S. AlMutairi]

On Tuesday, February 8, 2022 at 7:23:55 PM UTC+3 Mark H. Wood wrote:

Well, Ubuntu Focal has Solr in its package repository, but it's a
decade old, so Ubuntu is one of the distro.s on which you need to do
the installation manually. That means you'll need to create a user to
run it, very like Gentoo which I use.

I called that user 'solr'. I unpacked the Solr v8 files in
/opt/solr-8, created /var/solr-8 to hold the configuration and cores,
created /etc/solr-8 to hold the solr.in.sh and log4j2.xml, created
/var/log/solr-8 to hold the logs, and edited solr.in.sh to set
LOG4J_PROPS, SOLR_PID_DIR, SOLR_HOME, and SOLR_LOGS_DIR
appropriately. I set the /var/lib/solr-8 directory and contents to be
owned by 'solr'. I created an initscript (Ubuntu will need a "system
unit" or some such) to start /opt/solr-8/bin/solr as user 'solr'
running in ${SOLR_HOME}, with SOLR_INCLUDE set to
'/etc/solr-8/solr.in.sh' and passing options '-p 8983 -s
${SOLR_HOME}'. Much of this is what one does for any daemon that
isn't in the package manager.

At that point Solr should start and run, with no cores. I created a
directory ${SOLR_HOME}/ds7 and copied the DSpace 7 empty cores to
there, setting them owned by 'solr'. (You can put the cores directly
into ${SOLR_HOME} but I like to keep related cores organized this
way.)

I see why you doing it manually. Gentoo is not included as one of the supported OS's ( Debian, RedHat, The late CentOS, SuSE and Darwin are the supported ones ). By using the bundled installation script it will do just all of that. In Ubuntu it will create the Solr folders in accordance with the default Ubuntu filesystem hierarchy and will sets the main Solr environment variables as it seen below where the cores by default can be placed/copied into $SOLR_HOME or $SOLR_HOME/configsets ... Solr and the Dspace 4 cores installation are perfectly fine. I think the issue is with the solr-reindex-statistics script it's still referencing the statistics core in it's original location and that's should explain why I had to add that location as an allowed path to satisfy Solr ( https://solr.apache.org/guide/8_11/format-of-solr-xml.html. ). I'm totally new to dspace  ( Three months ). I was surprised seeing your comment that Dspace services by design can be isolated and the communication between the services should happen over the network and they do not need direct

access to each others' files.

# /etc/default/solr.in.sh

SOLR_PID_DIR="/var/solr"
SOLR_HOME="/var/solr/data"
LOG4J_PROPS="/var/solr/log4j2.xml"
SOLR_LOGS_DIR="/var/solr/logs"
SOLR_PORT="8983"

#SOLR_OPTS="$SOLR_OPTS -Dsolr.allowPaths=/opt/dspace/solr/statistics,/opt/dspace/temp/solr-data"

With reference to the dspace-tech discussion linked above: I don't
know where Solr is getting that path '/dspace/solr/statistics'. We
would need to see how Solr is configured at that site. Assuming that
DSpace is installed at '/dspace', then '/dspace/solr/*' should have
been copied into ${SOLR_HOME} and the copies set owned by the user
that runs Solr. The copies in '/dspace' are just templates, and are
not referenced by DSpace or Solr in use.

 

It's not specific to how Solr installed on Chris server. It can be easily replicated on any Linux distro by excluding [dspace]/solr/statistics from Solr Allowed paths and running [dspace]/bin/dspace solr-reindex-statistics ( I confirm this issue still exist in Dspace 7.2 ).

[Kazi Noman]

Dear all

I am also facing the same problem in DSpacd 7.2 on Debian 10.

The error shows:

                                error.top-level-communities.

Please help in this regard.

Thanks in advance..

Best, Kazi Noman

[Mohammad S. AlMutairi]

Hello Judge,

Solr installation in Debian is identical to Ubuntu installation. To fix the issue you are facing Just follow the steps you see below and avoid running [dspace]/bin/dspace solr-reindex-statistics ... It's broken. See the errors it generates when run in Debian attached.

1) mkdir /build
2) cd /build
3) wget https://downloads.apache.org/lucene/solr/8.11.1/solr-8.11.1.tgz
4) tar xzf solr-8.11.1.tgz solr-8.11.1/bin/install_solr_service.sh --strip-components=2

5) bash ./install_solr_service.sh solr-8.11.1.tgz -f
6) cp -r /opt/dspace/solr/* /var/solr/data/ # Do this step after installing dspace backend (REST API server). You need to change /opt/dspace to the folder you installed dspace backend into.
7) chown -R solr:solr /var/solr/data/
8) systemctl enable solr
9) systemctl restart solr

# Double check dspace cores:

10) curl http://localhost:8983/solr/admin/cores

[dspace]/bin/dspace solr-reindex-statistics error in Debian 10. It's still referencing the statistics core in dspace installation folder.

Respect to you all. Exit