[Dspace-tech] Error when importing statistics to elasticsearch in DSpace 5.1

[euler]

Dear all,

I am trying to import solr-format converted log files into Elastic Search
statistics but I'm having this error when running "dspace
stats-log-importer-elasticsearch -i D:\converted_logs\new_dspace.log.2015 -m
-s -v":

Exception: None of the configured nodes are available: []
org.elasticsearch.client.transport.NoNodeAvailableException: None of the
configured nodes are available: []
at
org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:278)
at
org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:197)
at
org.elasticsearch.client.transport.support.InternalTransportIndicesAdminClient.execute(InternalTransportIndicesAdminClient.java:86)
at
org.elasticsearch.client.support.AbstractIndicesAdminClient.refresh(AbstractIndicesAdminClient.java:418)
at
org.elasticsearch.action.admin.indices.refresh.RefreshRequestBuilder.doExecute(RefreshRequestBuilder.java:48)
at
org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:91)
at
org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:65)
at
org.dspace.statistics.util.StatisticsImporterElasticSearch.main(StatisticsImporterElasticSearch.java:380)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at
org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:225)
at
org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:77)

I am using windows 7 but also encountered this error in windows server 2003.
I have no problems when I imported my converted log files into solr
statistics using the command "dspace stats-log-importer". I also followed
the steps to enable and configure Elasticsearch mentioned here:
Elasticsearch Usage Statistics
<https://wiki.duraspace.org/display/DSDOC5x/Elasticsearch+Usage+Statistics>
. I can't find in the documentation about the settings in
elastic-search-statistics.cfg. I tried to uncomment all the properties there
and allowed port 9300 in my firewall. I also noticed that when I enabled
Elasticsearch, the server is not loading anything. Below is portion of my
dspace.log when I enabled Elasticsearch. The log just stopped there.

2015-04-07 17:12:38,537 INFO org.dspace.core.ConfigurationManager @ Loading
from classloader: file:/C:/DSpace/config/dspace.cfg
2015-04-07 17:12:38,572 INFO org.dspace.core.ConfigurationManager @ Using
dspace provided log configuration (log.init.config)
2015-04-07 17:12:38,574 INFO org.dspace.core.ConfigurationManager @
Loading: C:/DSpace/config/log4j.properties
2015-04-07 17:12:58,332 INFO
org.dspace.statistics.util.StatisticsImporterElasticSearch @ Getting
ElasticSearch Transport Client for StatisticsImporterElasticSearch...
2015-04-07 17:12:58,336 ERROR org.dspace.statistics.ElasticSearchLogger @
getClient reports null client
2015-04-07 17:12:58,339 INFO org.dspace.statistics.ElasticSearchLogger @
Creating TransportClient to [Address:127.0.0.1] [Port:9300]
[cluster.name:dspacestatslogging]
2015-04-07 17:12:58,653 INFO net.sf.ehcache.util.UpdateChecker @ New
update(s) found: 2.4.7
[http://www.terracotta.org/confluence/display/release/Release+Notes+Ehcache+Core+2.4]
2015-04-07 17:13:04,252 INFO org.elasticsearch.plugins @ [Buzz] loaded [],
sites []

Any advice please?

Thanks in advance,
euler



--
View this message in context: http://dspace.2283337.n4.nabble.com/Error-when-importing-statistics-to-elasticsearch-in-DSpace-5-1-tp4677334.html
Sent from the DSpace - Tech mailing list archive at Nabble.com.

[Peter Dietz]

Hi Euler,

The issue looks like this command line operation couldn't find an Elastic Search node. Is tomcat running? In the out-of-the-box architecture, DSpace will run an Elastic Search server within tomcat. So, ensure you have elasticsearch enabled, and then visit a dspace object such that a usage event is triggered. (View an item). The logs should show that an Elastic Search instance was started. Once that node is up, then you can run this data import.

A more complicate architecture, that works well for sites with larger infrastructure, is to have a dedicated elastic search server, and then configure the dspace/config/elasticsearch.cfg to use that IP address. Then, the command line operation should work, regardless of the tomcat setup. 

________________
Peter Dietz
Longsight
www.longsight.com
pe...@longsight.com
p: 740-599-5005 x809

------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
DSpace-tech mailing list
DSpac...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette

[euler]

Hi Peter,

Yes, tomcat is running and I'm monitoring tomcat's log for any errors but
none so far. I tried to enable Elastic Search, following your advice to
visit an item such that a usage event would be triggered (I did not attempt
to import logs to elasticsearch). Looking at my dspace logs, I only saw one
error:

2015-04-09 09:13:28,729 ERROR org.dspace.statistics.ElasticSearchLogger @
getCli
ent reports null client
2015-04-09 09:13:30,205 INFO org.dspace.statistics.ElasticSearchLogger @
Create
a nodeClient, allows transport clients to connect
2015-04-09 09:13:31,235 INFO org.elasticsearch.node @ [Fang]
version[1.4.0], pi
d[4616], build[bc94bd8/2014-11-05T14:26:12Z]
2015-04-09 09:13:31,235 INFO org.elasticsearch.node @ [Fang] initializing
...
2015-04-09 09:13:31,276 INFO org.elasticsearch.plugins @ [Fang] loaded [],
site
s [] <-- My dspace logs just stopped here, unless I restart Tomcat or run
any dspace commands

It seems that the elasticsearch.node is just initializing forever, and
nothing is loaded in my browser when I visit any pages.

Anything that I missed? I did try running an elasticsearch server from
another machine and point the IP in my elastic-search-statistics.cfg to that
machine, but the results of my dspace log is the same and nothing is loaded
in my browser. I would like to try to make elasticsearch work first, without
setting up an external elastic search server. I wonder if this is a windows
issue and would like to ask from the list if any windows users have
successfully enabled their elastic search.

Thanks in advance and regards,
euler



--
View this message in context: http://dspace.2283337.n4.nabble.com/Error-when-importing-statistics-to-elasticsearch-in-DSpace-5-1-tp4677334p4677377.html