Hello,
We have successfully installed Datafari 4.2.1 on a Debian 9.8 with 8GB RAM and Intel Core i5 Processor.
The goal is to index and search through different local NAS holding a lot of big video files (from 500Mo to 50Go per file) and data files (txt, excel, etc.)
When we mount a smb share on the debian hosting datafari, we can create a File system type Repository and a job to crawl it. It seems to be working quite fast, but as I read in the doc, this kind of repository is for testing purposes and it's not very convenient for what we need to do.
But when we try to create a "Windows shares" repository (smb share) and a job to crawl it, we run into several problems :
The Repository Connection status says "Connection working", but :
1/ Datafari is copying a lot of data in the /tmp folder. Is it copying every video file there? When the server is full, it flushes the tmp folder, but indexation in then very very slow.
2/ After a while, the whole system is failing :
We end with Giga octets of logs in the /opt/datafari/mcf/mcf_home/nohup.out file with thousands of messages like :
[main-SendThread(localhost:2181)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
[main-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/
127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
or
[Crawler idle cleanup thread-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/
127.0.0.1:2181. Will not attempt to authentic
ate using SASL (unknown error)
[Worker thread '27'-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/
127.0.0.1:2181. Will not attempt to authenticate using
SASL (unknown error)
[Worker thread '27'-SendThread(localhost:2181)] WARN org.apache.zookeeper.ClientCnxn - Session 0x16c72292007004a for server null, unexpected error, closing socket connection and attemptin
g reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
or...
The question is simple : What have we done wrong? :)
Subsidiary question : What do you need to examine this problem (config files, etc?)
Thanks for your help!