Start scylla problem

[jianjun396@gmail.com]

I created a VM and installed centos 7.

Downloaded the latest 0.15 version, followed the instruction to start scylla, but doesn't work.

After 

systemctl start scylla-server

systemctl status scylla-server

shows it is active.

but cqlsh gives error.

Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})

curl http://localhost:10000/storage_service/release_version

also gives error

curl: (7) Failed connect to localhost:10000; Connection refused

If I run scylla manually, it hangs even for:

scylla --help-loggers

How can I find out what's wrong? 

[Asias He]

Check the following:

What does  $ journalctl |grep scylla, say?

Is there any coredump in /var/lib/scylla/coredump?

Is scylla process still running, $ ps aux | grep scylla ?

Are you running on XFS?

--
You received this message because you are subscribed to the Google Groups "ScyllaDB users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-user...@googlegroups.com.
To post to this group, send email to scyllad...@googlegroups.com.
Visit this group at https://groups.google.com/group/scylladb-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/d10dd920-19ea-4d69-a466-24b42604324e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Asias

[jianjun396@gmail.com]

 journalctl |grep scylla

  Jan 18 22:29:44 localhost.localdomain sudo[33309]:     root : TTY=unknown ; PWD=/ ; USER=scylla ; COMMAND=/usr/bin/scylla --log-to-syslog 1 --log-to-stdout 0 --default-log-level info --developer-mode true --network-stack posix

No coredump.

ps aux | grep scylla

root      33309  0.0  0.0 196000  2816 ?        Ss   22:29   0:00 sudo -E -u scylla /usr/bin/scylla --log-to-syslog 1 --log-to-stdout 0 --default-log-level info --developer-mode true --network-stack posix

scylla    33315 99.6  0.3 17180007676 12688 ?   R    22:29   2:38 /usr/bin/scylla --log-to-syslog 1 --log-to-stdout 0 --default-log-level info --developer-mode true --network-stack posix

I'm not running on xfs, so I turned developer-mode on.

[Avi Kivity]

On 01/19/2016 07:44 AM, jianj...@gmail.com wrote:

I created a VM and installed centos 7.

How much memory does your VM have?  Prior to version 0.16, Scylla has problems with less that 2GB of memory.

[jianjun396@gmail.com]

The vm got 4G memory.

[Avi Kivity]

What does 'ss -ltnp' show?

To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/de3e7a61-ec84-4115-b604-d985854a9613%40googlegroups.com.

[Joel Jacobson]

Looks like the service is running. Try passing the host: ``cqlsh -h $IP ``

I would also take a look at nodetool status. ``nodetool -h $iP status``

[jianjun396@gmail.com]

I use localhost. cqlsh localhost or cqlsh 127.0.0.1.

None of it works.

ss -ltnp

State      Recv-Q Send-Q                         Local Address:Port                                        Peer Address:Port              

LISTEN     0      5                              192.168.122.1:53                                                     *:*                  

LISTEN     0      128                                        *:22                                                     *:*                  

LISTEN     0      128                                127.0.0.1:631                                                    *:*                  

LISTEN     0      100                                127.0.0.1:25                                                     *:*                  

LISTEN     0      128                                       :::22                                                    :::*                  

LISTEN     0      128                                      ::1:631                                                   :::*                  

LISTEN     0      100                                      ::1:25                                                    :::*   

I upgrade to 0.16 version. Now 

systemctl start scylla-server

hangs.

The virtual memory of scylla shows 16T.

[Asias He]

This means scylla does not boots up successfully. It does not listen on the cql port. 

 

I upgrade to 0.16 version. Now 

systemctl start scylla-server

hangs.

What does 

journalctl |grep scylla

say with 0.16 ?

 

The virtual memory of scylla shows 16T.

This is fine. It is not the memory scylla really consumes, it is just the virtual memory. 

 

--
You received this message because you are subscribed to the Google Groups "ScyllaDB users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-user...@googlegroups.com.
To post to this group, send email to scyllad...@googlegroups.com.
Visit this group at https://groups.google.com/group/scylladb-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/c7e61490-cb74-42e3-ad82-22f937dd2ea5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Asias

[Jianjun Wen]

journalctl |grep scylla

Jan 21 22:30:17 localhost.localdomain sudo[98640]:   scylla : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/lib/scylla/scylla_prepare

--

-Jianjun Wen

Wancube.com - 3D photography

Phone: 408 888 7023

[Asias He]

On Fri, Jan 22, 2016 at 2:32 PM, Jianjun Wen <jianj...@gmail.com> wrote:

journalctl |grep scylla

Jan 21 22:30:17 localhost.localdomain sudo[98640]:   scylla : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/lib/scylla/scylla_prepare

This looks bogus. At least we are supposed to see /usr/bin/scylla was started to run and some early messages printed by scylla. Perhaps scylla_prepare never completes for some reason. 

Takuya?

--

Asias

[Asias He]

On Fri, Jan 22, 2016 at 3:00 PM, Asias He <as...@scylladb.com> wrote:

On Fri, Jan 22, 2016 at 2:32 PM, Jianjun Wen <jianj...@gmail.com> wrote:

journalctl |grep scylla

Jan 21 22:30:17 localhost.localdomain sudo[98640]:   scylla : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/lib/scylla/scylla_prepare

This looks bogus. At least we are supposed to see /usr/bin/scylla was started to run and some early messages printed by scylla. Perhaps scylla_prepare never completes for some reason. 

Takuya?

 

In scylla-server.service,

  0 [Unit]

  1 Description=Scylla Server

  2 After=network.target libvirtd.service

Are network.target and libvirtd.service started properly?

--

Asias

[Jianjun Wen]

I manually run scylla_prepare, finished quickly

[jianjun396@gmail.com]

Found the problem:

 hwloc_topology_load hangs on my vm.

lstopo hangs also.

Is this an known issue to centos7?

I'm using the latest centos7 version.

To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-users+unsubscribe@googlegroups.com.
To post to this group, send email to scylladb-users@googlegroups.com.

Visit this group at https://groups.google.com/group/scylladb-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/c7e61490-cb74-42e3-ad82-22f937dd2ea5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Asias

--

-Jianjun Wen

Wancube.com - 3D photography

Phone: 408 888 7023

--

Asias

[Avi Kivity]

Ah, good progress.

This is not a known problem, on centos or otherwise.

Can you try an 'strace lstopo' to see what might be broken?

What hypervisor are you using?

To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-user...@googlegroups.com.
To post to this group, send email to scyllad...@googlegroups.com.

Visit this group at https://groups.google.com/group/scylladb-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/1975a6b3-debf-4fb1-8810-194088edf92c%40googlegroups.com.

[Jianjun Wen]

Tried strace, nothing useful.

I'm using vmware.

To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-user...@googlegroups.com.
To post to this group, send email to scyllad...@googlegroups.com.
Visit this group at https://groups.google.com/group/scylladb-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/c7e61490-cb74-42e3-ad82-22f937dd2ea5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Asias

--

-Jianjun Wen

Wancube.com - 3D photography

Phone: 408 888 7023

--

Asias

--
You received this message because you are subscribed to the Google Groups "ScyllaDB users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-user...@googlegroups.com.
To post to this group, send email to scyllad...@googlegroups.com.
Visit this group at https://groups.google.com/group/scylladb-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/1975a6b3-debf-4fb1-8810-194088edf92c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Pekka Enberg]

On Sat, Jan 30, 2016 at 7:14 PM, Jianjun Wen <jianj...@gmail.com> wrote:
> Tried strace, nothing useful.

Does setting the HWLOC_DEBUG_VERBOSE environment variable to 1 make
hwloc produce any debugging info?

- Pekka

[Jianjun Wen]

I worked with Holocaust team. They found it is VMware problem. Hwloc pushed an fix.  It works good for me.

[Avi Kivity]

Cool, let's hope they release a new version soon and that it gets propagated into the various distributions.