Elastic search timeout exceeded while starting Janusgraph

517 views
Skip to first unread message

troj...@gmail.com

unread,
Nov 21, 2017, 2:52:51 AM11/21/17
to JanusGraph users
When I start janusgraph by: bin/janusgraph start, everything is OK.The problem occurs when I start janusgraph at a system startup. I use Ubuntu server 16.04 and I added above command to the script in /etc/init.d as a result (service janusgraph status) I get:

Nov 21 08:40:36 vps423681.ovh.net systemd[1]: Starting LSB: Example initscript...
Nov 21 08:40:36 vps423681.ovh.net janusgraph[1414]: basename: missing operand
Nov 21 08:40:36 vps423681.ovh.net janusgraph[1414]: Try 'basename --help' for more information.
Nov 21 08:40:36 vps423681.ovh.net systemd[1]: Started LSB: Example initscript.
Nov 21 08:40:37 vps423681.ovh.net janusgraph[1414]: Forking Cassandra...
Nov 21 08:40:49 vps423681.ovh.net janusgraph[1414]: Running `nodetool statusthrift`... OK (returned exit status 0 and printed string "running")
Nov 21 08:40:50 vps423681.ovh.net janusgraph[1414]: Forking Elasticsearch...
Nov 21 08:41:51 vps423681.ovh.net janusgraph[1414]: Connecting to Elasticsearch (127.0.0.1:9200)............................. timeout exceeded
Nov 21 08:41:51 vps423681.ovh.net janusgraph[1414]: See /home/vps01/janusgraph-0.2.0-hadoop2/bin/../log/elasticsearch.log for Elasticsearch log


In elastic search logs I have:

[2017-11-21T08:12:07,299][INFO ][o.e.n.Node               ] [GBlVv_O] stopping ...
[2017-11-21T08:12:07,853][INFO ][o.e.n.Node               ] [GBlVv_O] stopped
[2017-11-21T08:12:07,856][INFO ][o.e.n.Node               ] [GBlVv_O] closing ...
[2017-11-21T08:12:07,999][INFO ][o.e.n.Node               ] [GBlVv_O] closed


I don't understand why I can't start janusgraph at the system startup but I can run it as a normal login user.

Robert Dale

unread,
Nov 21, 2017, 8:54:56 AM11/21/17
to troj...@gmail.com, JanusGraph users
Are you running it as a non-root user?

"Note for security reasons Elasticsearch and therefore janusgraph.sh must be run under a non-root account."


Robert Dale

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/7f2a3e87-2f8a-40fb-9522-caf3487b0901%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

troj...@gmail.com

unread,
Nov 21, 2017, 9:31:56 AM11/21/17
to JanusGraph users
I run it as a root.
I changed the startup script to run it as a normal user but there is still the same problem

Nov 21 15:25:08 vps423681.ovh.net su[1455]: pam_unix(su:session): session opened for user vps01 by (uid=0)
Nov 21 15:25:09 vps423681.ovh.net janusgraph[1413]: bash: start: command not found
Nov 21 15:25:09 vps423681.ovh.net janusgraph[1413]: basename: missing operand
Nov 21 15:25:09 vps423681.ovh.net janusgraph[1413]: Try 'basename --help' for more information.
Nov 21 15:25:09 vps423681.ovh.net systemd[1]: Started LSB: Example initscript.
Nov 21 15:25:10 vps423681.ovh.net janusgraph[1413]: Forking Cassandra...
Nov 21 15:25:22 vps423681.ovh.net janusgraph[1413]: Running `nodetool statusthrift`... OK (returned exit status 0 and printed string
Nov 21 15:25:22 vps423681.ovh.net janusgraph[1413]: Forking Elasticsearch...
Nov 21 15:26:23 vps423681.ovh.net janusgraph[1413]: Connecting to Elasticsearch (127.0.0.1:9200)............................. timeou
Nov 21 15:26:23 vps423681.ovh.net janusgraph[1413]: See /home/vps01/janusgraph-0.2.0-hadoop2/bin/../log/elasticsearch.log for Elasti

When I run modified(runs as a normal user) script as a root:
sudo /etc/init.d/janusgraph

The Janusgraph starts correctly:

Running `nodetool statusthrift`.. OK (returned exit status 0 and printed string "running").
Forking Elasticsearch...
Connecting to Elasticsearch (127.0.0.1:9200)....... OK (connected to 127.0.0.1:9200).
Forking Gremlin-Server...
Connecting to Gremlin-Server (127.0.0.1:8182).... OK (connected to 127.0.0.1:8182).
Run gremlin.sh to connect.

Tomek Rojek

On Tuesday, November 21, 2017 at 2:54:56 PM UTC+1, Robert Dale wrote:
Are you running it as a non-root user?

"Note for security reasons Elasticsearch and therefore janusgraph.sh must be run under a non-root account."


Robert Dale

On Tue, Nov 21, 2017 at 2:52 AM, <troj...@gmail.com> wrote:
When I start janusgraph by: bin/janusgraph start, everything is OK.The problem occurs when I start janusgraph at a system startup. I use Ubuntu server 16.04 and I added above command to the script in /etc/init.d as a result (service janusgraph status) I get:

Nov 21 08:40:36 vps423681.ovh.net systemd[1]: Starting LSB: Example initscript...
Nov 21 08:40:36 vps423681.ovh.net janusgraph[1414]: basename: missing operand
Nov 21 08:40:36 vps423681.ovh.net janusgraph[1414]: Try 'basename --help' for more information.
Nov 21 08:40:36 vps423681.ovh.net systemd[1]: Started LSB: Example initscript.
Nov 21 08:40:37 vps423681.ovh.net janusgraph[1414]: Forking Cassandra...
Nov 21 08:40:49 vps423681.ovh.net janusgraph[1414]: Running `nodetool statusthrift`... OK (returned exit status 0 and printed string "running")
Nov 21 08:40:50 vps423681.ovh.net janusgraph[1414]: Forking Elasticsearch...
Nov 21 08:41:51 vps423681.ovh.net janusgraph[1414]: Connecting to Elasticsearch (127.0.0.1:9200)............................. timeout exceeded
Nov 21 08:41:51 vps423681.ovh.net janusgraph[1414]: See /home/vps01/janusgraph-0.2.0-hadoop2/bin/../log/elasticsearch.log for Elasticsearch log


In elastic search logs I have:

[2017-11-21T08:12:07,299][INFO ][o.e.n.Node               ] [GBlVv_O] stopping ...
[2017-11-21T08:12:07,853][INFO ][o.e.n.Node               ] [GBlVv_O] stopped
[2017-11-21T08:12:07,856][INFO ][o.e.n.Node               ] [GBlVv_O] closing ...
[2017-11-21T08:12:07,999][INFO ][o.e.n.Node               ] [GBlVv_O] closed


I don't understand why I can't start janusgraph at the system startup but I can run it as a normal login user.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.

Robert Dale

unread,
Nov 21, 2017, 9:40:49 AM11/21/17
to troj...@gmail.com, JanusGraph users
Since you can start it just fine with sudo it would imply that the janusgraph.sh script is good to go.
Maybe the error lies in how systemd is calling it.
I see other errors in your script as well e.g. `bash: start: command not found`.
Are you able to share the systemd config and janusgraph.sh?


Robert Dale

To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/7dd35379-72e2-458b-80a8-703f82014509%40googlegroups.com.

troj...@gmail.com

unread,
Nov 21, 2017, 9:53:55 AM11/21/17
to JanusGraph users
I don't know what do you mean by systemd config. My janusgraph from /etc/init.d looks:

#!/bin/sh
# kFreeBSD do not accept scripts as interpreters, using #!/bin/sh and sourcing.
if [ true != "$INIT_D_SCRIPT_SOURCED" ] ; then
   
set "$0" "$@"; INIT_D_SCRIPT_SOURCED=true . /lib/init/init-d-script
fi
### BEGIN INIT INFO
# Provides:          janusgraph
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Example initscript
# Description:       This file should be used to construct scripts to be
#                    placed in /etc/init.d.  This example start a
#                    single forking daemon capable of writing a pid
#                    file.  To get other behavoirs, implemend
#                    do_start(), do_stop() or other functions to
#                    override the defaults in /lib/init/init-d-script.
### END INIT INFO

# DESC="Janus graph database"
# DAEMON=/home/vps01/janusgraph-0.2.0-hadoop2/bin/janusgraph.sh start
su vps01
-c "bash -c '/home/vps01/janusgraph-0.2.0-hadoop2/bin/janusgraph.sh start & $1'"

Tomek

Robert Dale

Robert Dale

unread,
Nov 21, 2017, 9:29:09 PM11/21/17
to troj...@gmail.com, JanusGraph users

Elasticsearch currently requires the working directory to be the JanusGraph install dir.  Try:
su vps01 -c "cd /home/vps01/janusgraph-0.2.0-hadoop2; ./bin/janusgraph.sh start"


However, with systemd, it may be preferable to create a systemd unit file.

Copy the service template below to `/etc/systemd/system/janusgraph.service` and update the paths `/path/to/janusgraph` with the actual install path of JanusGraph.

[Unit]
Description=JanusGraph
After=network.target

[Service]
Type=forking
User=<username>
Group=<groupname>
WorkingDirectory=/path/to/janusgraph-0.2.0-hadoop2
# Using /bin/bash gets around potential SELinux issues with Java
ExecStart=/bin/bash -c '/path/to/janusgraph-0.2.0-hadoop2/bin/janusgraph.sh -v start'
ExecStop=/bin/bash -c '/path/to/janusgraph-0.2.0-hadoop2/bin/janusgraph.sh stop'

[Install]
WantedBy=multi-user.target


Enable the service with `systemctl enable janusgraph`

Start the service with `systemctl start janusgraph`



Robert Dale

To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/d025b8d6-7b2f-4bf6-a297-73e394f72163%40googlegroups.com.

troj...@gmail.com

unread,
Nov 23, 2017, 6:14:58 AM11/23/17
to JanusGraph users
I didn't try to update /etc/init.d/janusgraph. I created systemd service file and it definitely works without any problem. The only difference is that I don't see 3 processes (cassandra, gremlin and elasticsearch) in the process list (ps -ef).
Thank you Robert.

Tomek

Robert Dale

Reply all
Reply to author
Forward
0 new messages