Installation authentication problem

[Ian Stewart]

Hi all

I am having some installation problems and wonder if anyone can advise. I have followed the installation guide or community edition diligently and have almost all of the components of datafari working. However, Datafari does not seem to be able to talk to manifoldcf through either the API or when I use the web interface. I get the attached error and the following message in the manifoldcf log. 

ERROR 2018-08-07T09:05:09,671 (Agents thread) - MCF|MCF-agent|apache.manifoldcf.agents|Exception tossed: Error getting connection: FATAL: pass$

org.apache.manifoldcf.core.interfaces.ManifoldCFException: Error getting connection: FATAL: password authentication failed for user "manifoldc$

        at org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:83) ~[mcf-core.jar:?]

        at org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:797) ~[mcf-core.jar:?]

        at org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1457) ~[mcf-core.jar:?]

        at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:146) ~[mcf-core.jar:?]

        at org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:204) ~[mcf-core.jar:?]

        at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:837) ~[mcf-core.jar:?]

        at org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseTable.java:221) ~[mcf-core.jar:?]

        at org.apache.manifoldcf.agents.agentmanager.AgentManager.getAllAgents(AgentManager.java:218) ~[mcf-agents.jar:?]

        at org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:261) ~[mcf-agents.jar:?]

        at org.apache.manifoldcf.agents.system.AgentsDaemon$AgentsThread.run(AgentsDaemon.java:209) [mcf-agents.jar:?]

Caused by: org.postgresql.util.PSQLException: FATAL: password authentication failed for user "manifoldcf"

        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:438) ~[postgresql-42.1.3.jar:42.1.3]

        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222) ~[postgresql-42.1.3.jar:42.1.3]

        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49) ~[postgresql-42.1.3.jar:42.1.3]

        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194) ~[postgresql-42.1.3.jar:42.1.3]

        at org.postgresql.Driver.makeConnection(Driver.java:450) ~[postgresql-42.1.3.jar:42.1.3]

        at org.postgresql.Driver.connect(Driver.java:252) ~[postgresql-42.1.3.jar:42.1.3]

        at java.sql.DriverManager.getConnection(DriverManager.java:664) ~[?:1.8.0_171]

        at java.sql.DriverManager.getConnection(DriverManager.java:247) ~[?:1.8.0_171]

        at org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:144) ~[mcf-core.jar:?]

        at org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:126) ~[mcf-core.jar:?]

        at org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:75) ~[mcf-core.jar:?]

        ... 9 more

Looking at the logs, this appears to be a psql account. After getting the error, i manually created a postgres database called manifoldcf with a user called manifoldcf. However, the error continues. I also updated the password string in properties-global.xml using the  obfuscation tool so that I could be sure it matched.

While I am assuming its a psql issue, I cannot find anywhere in the code base that lists a 'user' manifold. As such, I might be looking in the wrong place. 

I am a bit stumped. 

Many thanks 

[Olivier Tavard]

Hi,

Could you be more specific about your installation please ? Is it Datafari 4.1 or did you build from the source code ? Did you install it on Debian ? What is the version of the OS ? What are the ressources of the VM/server ?

Thanks !

Olivier

[Ian Stewart]

Thanks for response. 

I downloaded the latest deb from datafari https://www.datafari.com/en/download.html. I'm using ubuntu 16 and did not compile from source. I set up a new VM instance just for datafari after hitting an unrelated stumbling block on my main server. It's 2 core with 16gb ram, so should be sufficient. 


best

[Olivier Tavard]

Hello,

Thanks for the details.

I can reproduce your issue on Ubuntu 16.04 also. Let me check it out and I will come back to you.

Thanks,

Olivier

--
You received this message because you are subscribed to the Google Groups "Datafari" group.
To unsubscribe from this group and stop receiving emails from it, send an email to datafari+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Olivier Tavard]

Hi,

It seems that your Java home variable is not settled correctly.

Could you enter : echo $JAVA_HOME in your terminal and see if it is empty or not.

Check also the file /opt/datafari/mcf/mcf_home/properties-global.xml, check the line :

<property name="org.apache.manifoldcf.dbsuperuserpassword.obfuscated" value=""/>

and see if the value is empty or not.

If it is the case, it means that the obfuscate script did not work during the installation of the Datafari package because the Java home variable was not correctly settled.

I will do some other tests but it seems not specific to Ubuntu thus.

Best regards,

Olivier

[Ian Stewart]

Hi. Thanks for this. I've now deleted my test instance but will reinstall. However, I think I can speak to this point. 

I set up datafari using the deb file on two instances. The first time around, it is true that the java_home variable was not set at the time of running the install script. I realised this was an issue and manually ran the obfuscation script and edited the properties-global.xml file to include the result as you have indicated. That did not solve the problem. 

I therefore set up a new server, exported the paths, and ran the installer. I was careful to test the java_home path to ensure it was correct. However, I did note that Datafari gave a warning to say that it was not set correctly. I had a look at the install script and manually tested the relevant line using a simple cd / cat command and it seemed to work. Again, however, the obfuscated password was blank. Again, manually running the obfuscation script using the pgsql user password did not solve the issue. 

best

To unsubscribe from this group and stop receiving emails from it, send an email to datafari+u...@googlegroups.com.

[Olivier Tavard]

Hello,

Thanks for your detailed answer.

My answers are below.

2018-08-08 13:18 GMT+02:00 Ian Stewart <ian.s...@gmail.com>:

Hi. Thanks for this. I've now deleted my test instance but will reinstall. However, I think I can speak to this point. 

I set up datafari using the deb file on two instances. The first time around, it is true that the java_home variable was not set at the time of running the install script. I realised this was an issue and manually ran the obfuscation script and edited the properties-global.xml file to include the result as you have indicated. That did not solve the problem. 

Yes because you need to edit the file /opt/datafari/bin/common/init_state.properties and change the value 'initialized' to 'installed' because Datafari needed to be reinitialized with the new value.

I therefore set up a new server, exported the paths, and ran the installer. I was careful to test the java_home path to ensure it was correct. However, I did note that Datafari gave a warning to say that it was not set correctly. I had a look at the install script and manually tested the relevant line using a simple cd / cat command and it seemed to work. Again, however, the obfuscated password was blank. Again, manually running the obfuscation script using the pgsql user password did not solve the issue. 

It can be the same problem if you already started Datafari and after edited the value in properties-global.xml ie Datafari needs to be reinitialized.

But I don' understand when you say that java home was correct but Datafari gave you a warning about it. Was the message  : "Environment variable JAVA_HOME is not properly set." ? If yes it means that the script did not find the java home variable.

Usually when we install Datafari we set java home in /etc/profile or /etc/environment (ie for all users). Because for installing Datafari, you need to be root so the java home variable must be correct for root user. And when you start Datafari with a non root user, the variable must be correct too for that user.

I tested it on Ubuntu 16.04 on AWS this morning and it was OK. I also patched Datafari source code(for next release) as I said in my previous message to oblige Datafari installation to stop if java home variable is not set so thanks for that for your report.

Anyway I realized that the installation documentation was not very clear so I will update the Datafari installation page to be more comprehensible.

Let me know if you have some issues again.

Best regards,

Olivier

 

To unsubscribe from this group and stop receiving emails from it, send an email to datafari+unsubscribe@googlegroups.com.

[Ian Stewart]

Thanks for your guidance, Oliver. I started afresh paying particular attention to this point and succeeded with a good install. 

[Olivier Tavard]

Hello,

I am glad that you succeeded to install Datafari.

If you have another issues, do no hesitate to ask us.

Best regards,

Olivier