Production Setup problems

[Leonardo Figueiredo]

Hello,

I'm having some problems putting my institution's repository into production,
and I was wondering if anyone could help me, as I don't know what else to check.

Firstly, the repository worked correctly with the instructions in the official documentation using an internal IP of the institution (accessible only on the institution's network. Let's assume that the internal IP is 10.7.2.70) with the following settings:

- local.cfg file (in /dspace/config/):

dspace.server.url = https://my.site/server

dspace.ui.url = https://my.site


 - config.prod.yml (in /home/dspace/dspace-frontend-prod/config/):

ui:
  ssl: false
  host: localhost
  port: 4000
  nameSpace: /

rest:
  ssl: true
  host: my.site
  port: 443
  nameSpace: /server

 - Apache2 configuration:
 
 
<VirtualHost *:443>
   
    ServerName my.site

    ProxyPreserveHost On
    RequestHeader set X-Forwarded-Proto https

    SSLEngine on
    SSLCertificateFile /etc/tomcat10/certificate/my-site_crt.crt
    SSLCertificateKeyFile /etc/tomcat10/certificate/mysite.key

    SSLCertificateChainFile /etc/tomcat10/certificate/intermediate.pem

    # Proxy all HTTPS requests to "/server" from Apache to Tomcat via AJP connector
    ProxyPass /server ajp://localhost:8009/server
    ProxyPassReverse /server ajp://localhost:8009/server

    #UI
    ProxyPass / http://localhost:4000/
    ProxyPassReverse / http://localhost:4000/

</VirtualHost>


As the site was accessible internally, I asked the sector that manages the institution's network for a public IP, and soon after I requested that in their DNS servers, the name "my.site" be pointed to the public IP, and this public IP
point to the Internal IP (10.7.2.70). However, I have no longer been able to access the website.

Apparently the error has occurred in Apache communication. Below are the error logs:

- /var/log/apache2/error.log:

[Mon Jul 29 11:32:05.836354 2024] [proxy:error] [pid 345333:tid 140116479678144] (111)Connection refused: AH00957: http: attempt to connect to 127.0.0.1:4000 (localhost:4000) failed
[Mon Jul 29 11:32:05.836460 2024] [proxy_http:error] [pid 345333:tid 140116479678144] [client 40.77.167.51:11411] AH01114: HTTP: failed to make connection to backend: localhost
[Mon Jul 29 11:32:05.860228 2024] [proxy:error] [pid 345492:tid 140115657619136] (111)Connection refused: AH00957: http: attempt to connect to 127.0.0.1:4000 (localhost:4000) failed
[Mon Jul 29 11:32:05.860310 2024] [proxy_http:error] [pid 345492:tid 140115657619136] [client 52.167.144.231:30290] AH01114: HTTP: failed to make connection to backend: localhost
[Mon Jul 29 11:32:06.461271 2024] [proxy:error] [pid 345492:tid 140115632441024] (111)Connection refused: AH00957: http: attempt to connect to 127.0.0.1:4000 (localhost:4000) failed
[Mon Jul 29 11:32:06.461365 2024] [proxy_http:error] [pid 345492:tid 140115632441024] [client 40.77.167.51:11393] AH01114: HTTP: failed to make connection to backend: localhost

When I run the "yarn test:rest" command, I get this message:

yarn test:rest
yarn run v1.22.22
$ ts-node --project ./tsconfig.ts-node.json scripts/test-rest.ts
Building production app config
Overriding app config with /home/dspace/dspace-angular-dspace-8.0/config/config.yml
Overriding app config with /home/dspace/dspace-angular-dspace-8.0/config/config.prod.yml
...Testing connection to REST API at https://my.site/server/api...

ERROR connecting to REST API
Error: connect ETIMEDOUT 200.239.66.59:443


When I run the "pm2 logs" command, I get these error messages:

3|dspace-ui  | ERROR Error: undefined doesn't contain the link sites
3|dspace-ui  |     at /home/dspace/dspace-frontend-prod/dist/server/main.js:1:630512
3|dspace-ui  |     at /home/dspace/dspace-frontend-prod/dist/server/main.js:1:3456119
3|dspace-ui  |     at OperatorSubscriber2._this._next (/home/dspace/dspace-frontend-prod/dist/server/main.js:1:3417975)
3|dspace-ui  |     at OperatorSubscriber2.Subscriber2.next (/home/dspace/dspace-frontend-prod/dist/server/main.js:1:3371524)
3|dspace-ui  |     at /home/dspace/dspace-frontend-prod/dist/server/main.js:1:3456106
3|dspace-ui  |     at OperatorSubscriber2._this._next (/home/dspace/dspace-frontend-prod/dist/server/main.js:1:3417975)
3|dspace-ui  |     at OperatorSubscriber2.Subscriber2.next (/home/dspace/dspace-frontend-prod/dist/server/main.js:1:3371524)
3|dspace-ui  |     at /home/dspace/dspace-frontend-prod/dist/server/main.js:1:3492953
3|dspace-ui  |     at OperatorSubscriber2._this._next (/home/dspace/dspace-frontend-prod/dist/server/main.js:1:3417975)
3|dspace-ui  |     at OperatorSubscriber2.Subscriber2.next (/home/dspace/dspace-frontend-prod/dist/server/main.js:1:3371524)
3|dspace-ui  | ERROR Error: undefined doesn't contain the link sites
3|dspace-ui  |     at /home/dspace/dspace-frontend-prod/dist/server/main.js:1:630512
3|dspace-ui  |     at /home/dspace/dspace-frontend-prod/dist/server/main.js:1:3456119
3|dspace-ui  |     at OperatorSubscriber2._this._next (/home/dspace/dspace-frontend-prod/dist/server/main.js:1:3417975)
3|dspace-ui  |     at OperatorSubscriber2.Subscriber2.next (/home/dspace/dspace-frontend-prod/dist/server/main.js:1:3371524)
3|dspace-ui  |     at /home/dspace/dspace-frontend-prod/dist/server/main.js:1:3456106
3|dspace-ui  |     at OperatorSubscriber2._this._next (/home/dspace/dspace-frontend-prod/dist/server/main.js:1:3417975)
3|dspace-ui  |     at OperatorSubscriber2.Subscriber2.next (/home/dspace/dspace-frontend-prod/dist/server/main.js:1:3371524)
3|dspace-ui  |     at /home/dspace/dspace-frontend-prod/dist/server/main.js:1:3492953
3|dspace-ui  |     at OperatorSubscriber2._this._next (/home/dspace/dspace-frontend-prod/dist/server/main.js:1:3417975)
3|dspace-ui  |     at OperatorSubscriber2.Subscriber2.next (/home/dspace/dspace-frontend-prod/dist/server/main.js:1:3371524)


I have checked other topics with similar situations and applied the suggestions, but unfortunately nothing has had any effect, and i no longer know what to check.

Could it be the case that there is some blocking that is preventing my internal IP from accessing the external IP?  Neither the machine nor the unit I work on has a firewall or any type of blocking.

Shouldn't the fact that the production configuration worked correctly while the DNS only pointed to the internal IP indicate that no further configuration would need to be done when a public IP pointing to the internal one was defined?

I would be very grateful if anyone else could give me a tip on where else to check.

PS: sorry for my English, I hope you can understand me well

[mw...@iu.edu]

On Mon, Jul 29, 2024 at 03:33:03PM +0000, Leonardo Figueiredo wrote:
> - /var/log/apache2/error.log:
>
> [Mon Jul 29 11:32:05.836354 2024] [proxy:error] [pid 345333:tid 140116479678144] (111)Connection refused: AH00957: http: attempt to connect to 127.0.0.1:4000 (localhost:4000) failed
> [Mon Jul 29 11:32:05.836460 2024] [proxy_http:error] [pid 345333:tid 140116479678144] [client 40.77.167.51:11411] AH01114: HTTP: failed to make connection to backend: localhost
> [Mon Jul 29 11:32:05.860228 2024] [proxy:error] [pid 345492:tid 140115657619136] (111)Connection refused: AH00957: http: attempt to connect to 127.0.0.1:4000 (localhost:4000) failed
> [Mon Jul 29 11:32:05.860310 2024] [proxy_http:error] [pid 345492:tid 140115657619136] [client 52.167.144.231:30290] AH01114: HTTP: failed to make connection to backend: localhost
> [Mon Jul 29 11:32:06.461271 2024] [proxy:error] [pid 345492:tid 140115632441024] (111)Connection refused: AH00957: http: attempt to connect to 127.0.0.1:4000 (localhost:4000) failed
> [Mon Jul 29 11:32:06.461365 2024] [proxy_http:error] [pid 345492:tid 140115632441024] [client 40.77.167.51:11393] AH01114: HTTP: failed to make connection to backend: localhost

"Connection refused" means that nothing is listening to port 4000 on
localhost. I would start by figuring out why. I would try to find
out what the front-end *is* listening for, and correct that if
necessary. Here 'netstat -tlnp | grep PM2' worked for me.

When certain that the front end is listening properly, if the problem
persists then I would next check for firewall REJECT rules that might
be blocking connections.

> PS: sorry for my English, I hope you can understand me well

Your English is quite good enough for me to understand.

--
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
library.indianapolis.iu.edu

[Leonardo Figueiredo]

(I believe I responded directly to Mark, and that's why the answer ended up not appearing here for everyone to see, sorry)

Hi, 

First of all, thanks for the help and tips. 

A few hours after making this post, I managed to speak personally with the team that manages the University's firewall, and after explaining my situation in more detail, they found that there was a network block in communication between the internal and external IP. After the situation was resolved, the website was live and available both internally and externally. 

 I was really out of ideas on what to check, trying to figure out what could be wrong with the system settings, but luckily everything is working fine now. 

 Thank you very much.