Possible authentication issue with Cassandra 1.2

1,001 views
Skip to first unread message

David van Geest

unread,
Apr 10, 2014, 2:12:24 PM4/10/14
to spring-dat...@googlegroups.com
At David's request, I'm starting a new thread to discuss authentication in Cassandra 1.2.

I enabled authentication by setting "authenticator: PasswordAuthenticator" in /etc/cassandra/cassandra.yaml. It does say "Please increase system_auth keyspace replication factor if you use this authenticator"... however I'm not sure where to change that, so I did not do anything there.

I then logged into Cassandra via cqlsh with the "cassandra" superuser and created an "al2_dev" user with password "password" (this is just a dev box). I am able to login with this user using cqlsh:

davidv@calydon ~ $ cqlsh -u al2_dev -p password
Connected to AL2 at localhost:9160.
[cqlsh 3.1.7 | Cassandra 1.2.10 | CQL spec 3.0.0 | Thrift protocol 19.36.0]
Use HELP for help.
cqlsh> exit

However, when using spring-cql 1.0.0.M1, I get an exception when Spring tries to create the cassandraCluster bean:

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cassandraCluster': Invocation of init method failed; nested exception is com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host localhost/127.0.0.1: java.lang.ArrayIndexOutOfBoundsException: 15
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1512)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:296)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:293)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:320)
... 75 more
Caused by: com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host localhost/127.0.0.1: java.lang.ArrayIndexOutOfBoundsException: 15
at com.datastax.driver.core.Connection.waitForSaslCompletion(Connection.java:203)
at com.datastax.driver.core.Connection.initializeTransport(Connection.java:169)
at com.datastax.driver.core.Connection.<init>(Connection.java:131)
at com.datastax.driver.core.Connection.<init>(Connection.java:59)
at com.datastax.driver.core.Connection$Factory.open(Connection.java:444)
at com.datastax.driver.core.ControlConnection.tryConnect(ControlConnection.java:205)
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:168)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:81)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:812)
at com.datastax.driver.core.Cluster$Manager.access$100(Cluster.java:739)
at com.datastax.driver.core.Cluster.<init>(Cluster.java:82)
at com.datastax.driver.core.Cluster.<init>(Cluster.java:67)
at com.datastax.driver.core.Cluster$Builder.build(Cluster.java:708)
at org.springframework.cassandra.config.CassandraCqlClusterFactoryBean.afterPropertiesSet(CassandraCqlClusterFactoryBean.java:186)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$6.run(AbstractAutowireCapableBeanFactory.java:1561)
at java.security.AccessController.doPrivileged(Native Method)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1559)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1509)
        ......

The relevant part of my XML config is:

  <bean id="cassandraAuthProvider" class="com.datastax.driver.core.sasl.DseAuthProvider" />

  <cql:cluster
    contact-points="${cassandra.contact_points}"
    port="${cassandra.port}"
    ssl-enabled="${cassandra.ssl_enabled}"
    auth-info-provider-ref="cassandraAuthProvider"
    username="${cassandra.username}"
    password="${cassandra.password}" />
  <cql:session keyspace-name="${cassandra.keyspace_name}" />
  <cql:template />

If I turn off authentication by going back to the "AllowAllAuthenticator" and restarting Cassandra, things work as expected.

Versions:

davidv@calydon ~ $ cassandra -v
xss =  -ea -javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms2949M -Xmx2949M -Xmn737M -XX:+HeapDumpOnOutOfMemoryError -Xss256k
1.2.10

(this is the DataStax distro)

davidv@calydon ~ $ java -version
java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)

spring-context: 3.2.8.RELEASE

(the rest of these should be unrelated I would think...)
spring-orm: 3.2.8.RELEASE
spring-security-crypto: 3.2.3.RELEASE
spring-data-couchbase: 1.0.0.RELEASE
spring-rabbit: 1.3.1.RELEASE

One note, I'm excluding spring-data-commons from the spring-cql dependencies because it conflicts with the spring-data-commons required by spring-data-couchbase. To ensure this wasn't causing the problem, I temporarily allowed spring-cql's spring-data-commons, but got the same result.

I am by no means a Cassandra or Spring expert, so it's quite possible I'm doing something incorrectly. Any ideas? Thanks!


David van Geest

unread,
Apr 10, 2014, 2:25:51 PM4/10/14
to spring-dat...@googlegroups.com
I should also perhaps add that authorization is not a factor here. I have "authorizer: AllowAllAuthorizer" set, and can access the keyspace in question:

davidv@calydon ~ $ cqlsh -u al2_dev -p password
Connected to AL2 at localhost:9160.
[cqlsh 3.1.7 | Cassandra 1.2.10 | CQL spec 3.0.0 | Thrift protocol 19.36.0]
Use HELP for help.
cqlsh> use al2_dev;
cqlsh:al2_dev> select * from readings;

 instrument_id | date_time                | value
---------------+--------------------------+-------
             5 | 2014-04-09 20:20:38-0400 |  12.5
             5 | 2014-04-09 21:20:38-0400 |  12.4
             5 | 2014-04-09 22:20:38-0400 |  12.3
             1 | 2014-04-09 22:20:38-0400 |  12.4
             2 | 2014-04-09 22:20:38-0400 |  12.9
             6 | 2014-04-09 20:20:38-0400 |  12.5
             6 | 2014-04-09 21:20:38-0400 |  12.4
             6 | 2014-04-09 22:20:38-0400 |  12.3



--
You received this message because you are subscribed to the Google Groups "Spring Data Cassandra" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spring-data-cass...@googlegroups.com.
To post to this group, send email to spring-dat...@googlegroups.com.
Visit this group at http://groups.google.com/group/spring-data-cassandra.
To view this discussion on the web visit https://groups.google.com/d/msgid/spring-data-cassandra/fa2cd320-173a-471f-ab0d-4f3cabcb078d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
David van Geest
Software Engineer
SpinDance, Inc.
616.355.7000 x113

David Webb

unread,
Apr 10, 2014, 3:07:48 PM4/10/14
to spring-dat...@googlegroups.com
Change the Authorizer as follows:

authorizer: org.apache.cassandra.auth.CassandraAuthorizer


Are you using DSE or C* OSS?

If you are not using DSE, try either omitting the auth-info-provider-ref, or using this class as the auth provider

com.datastax.driver.core.PlainTextAuthProvider

The DseAuthProvider is for DSE only.

HTH


David van Geest

unread,
Apr 10, 2014, 3:43:49 PM4/10/14
to David Webb, spring-data-cassandra
Thanks for the reply David!

I'm using DataStax Community Edition from http://debian.datastax.com/community. I assume this is the same as C* OSS.

I tried the following 4 things, both with AllowAllAuthorizer and org.apache.cassandra.auth.CassandraAuthorizer:

1. Removed the auth-info-provider-ref and removed the cassandraAuthProvider bean (also tried leaving in the cassandraAuthProvider bean, and also renaming it to authProvider)

All of the above give the following:

Caused by: com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host localhost/127.0.0.1: Host localhost/127.0.0.1 requires authentication, but no authenticator found in Cluster configuration
at com.datastax.driver.core.AuthProvider$1.newAuthenticator(AuthProvider.java:39)
at com.datastax.driver.core.Connection.initializeTransport(Connection.java:163)
at com.datastax.driver.core.Connection.<init>(Connection.java:131)
at com.datastax.driver.core.Connection.<init>(Connection.java:59)
at com.datastax.driver.core.Connection$Factory.open(Connection.java:444)
at com.datastax.driver.core.ControlConnection.tryConnect(ControlConnection.java:205)
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:168)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:81)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:812)
at com.datastax.driver.core.Cluster$Manager.access$100(Cluster.java:739)
at com.datastax.driver.core.Cluster.<init>(Cluster.java:82)
at com.datastax.driver.core.Cluster.<init>(Cluster.java:67)
at com.datastax.driver.core.Cluster$Builder.build(Cluster.java:708)
at org.springframework.cassandra.config.CassandraCqlClusterFactoryBean.afterPropertiesSet(CassandraCqlClusterFactoryBean.java:186)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$6.run(AbstractAutowireCapableBeanFactory.java:1561)
at java.security.AccessController.doPrivileged(Native Method)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1559)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1509)
... 82 more

2. Tried using the PlainTextAuthProvider as follows:

  <bean id="cassandraAuthProvider" class="com.datastax.driver.core.PlainTextAuthProvider" />

  <cql:cluster
    contact-points="${cassandra.contact_points}"
    port="${cassandra.port}"
    ssl-enabled="${cassandra.ssl_enabled}"
    auth-info-provider-ref="cassandraAuthProvider"
    username="${cassandra.username}"
    password="${cassandra.password}" />
  <cql:session keyspace-name="${cassandra.keyspace_name}" />
  <cql:template />

Got the following:

Caused by: org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.datastax.driver.core.PlainTextAuthProvider]: No default constructor found; nested exception is java.security.PrivilegedActionException: java.lang.NoSuchMethodException: com.datastax.driver.core.PlainTextAuthProvider.<init>()
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:83)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$4.run(AbstractAutowireCapableBeanFactory.java:1025)
at java.security.AccessController.doPrivileged(Native Method)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1023)
... 93 more
Caused by: java.security.PrivilegedActionException: java.lang.NoSuchMethodException: com.datastax.driver.core.PlainTextAuthProvider.<init>()
at java.security.AccessController.doPrivileged(Native Method)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:71)
... 96 more
Caused by: java.lang.NoSuchMethodException: com.datastax.driver.core.PlainTextAuthProvider.<init>()
at java.lang.Class.getConstructor0(Class.java:2810)
at java.lang.Class.getDeclaredConstructor(Class.java:2053)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy$1.run(SimpleInstantiationStrategy.java:73)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy$1.run(SimpleInstantiationStrategy.java:71)

3. Apparently PlainTextAuthProvider needs username/password constructor arguments. So, tried this:

  <bean id="cassandraAuthProvider" class="com.datastax.driver.core.PlainTextAuthProvider" >
    <constructor-arg index="0" value="${cassandra.username}"/>
    <constructor-arg index="1" value="${cassandra.password}"/>
  </bean>

  <cql:cluster
    contact-points="${cassandra.contact_points}"
    port="${cassandra.port}"
    ssl-enabled="${cassandra.ssl_enabled}"
    auth-info-provider-ref="cassandraAuthProvider" />
  <cql:session keyspace-name="${cassandra.keyspace_name}" />
  <cql:template />

This gives the original exception:

Caused by: com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host localhost/127.0.0.1: java.lang.ArrayIndexOutOfBoundsException: 15
at com.datastax.driver.core.Connection.waitForSaslCompletion(Connection.java:203)
at com.datastax.driver.core.Connection.initializeTransport(Connection.java:169)
at com.datastax.driver.core.Connection.<init>(Connection.java:131)
at com.datastax.driver.core.Connection.<init>(Connection.java:59)
at com.datastax.driver.core.Connection$Factory.open(Connection.java:444)
at com.datastax.driver.core.ControlConnection.tryConnect(ControlConnection.java:205)
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:168)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:81)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:812)
at com.datastax.driver.core.Cluster$Manager.access$100(Cluster.java:739)
at com.datastax.driver.core.Cluster.<init>(Cluster.java:82)
at com.datastax.driver.core.Cluster.<init>(Cluster.java:67)
at com.datastax.driver.core.Cluster$Builder.build(Cluster.java:708)
at org.springframework.cassandra.config.CassandraCqlClusterFactoryBean.afterPropertiesSet(CassandraCqlClusterFactoryBean.java:186)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$6.run(AbstractAutowireCapableBeanFactory.java:1561)
at java.security.AccessController.doPrivileged(Native Method)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1559)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1509)

4. I also tried 3, but left the username and password attributes in the cql:cluster element. Got the same results.

Ready and willing to try anything else :-).


--
You received this message because you are subscribed to the Google Groups "Spring Data Cassandra" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spring-data-cass...@googlegroups.com.
To post to this group, send email to spring-dat...@googlegroups.com.
Visit this group at http://groups.google.com/group/spring-data-cassandra.

For more options, visit https://groups.google.com/d/optout.

David Webb

unread,
Apr 10, 2014, 4:10:32 PM4/10/14
to spring-dat...@googlegroups.com
Is there anything in the C* log?

Are you sure your property placeholders are getting set properly?  Perhaps try putting the username and password in the XML config?

Not sure what else to try at this point.

David van Geest

unread,
Apr 10, 2014, 4:26:38 PM4/10/14
to spring-data-cassandra
Good ideas.

Tried putting the user/pass directly in the config, but got the same results.

I looked at the Cassandra log. Interestingly, the same exception shows up there:

ERROR 16:23:24,408 Unexpected exception during request
java.lang.ArrayIndexOutOfBoundsException: 15
        at org.apache.cassandra.transport.Message$Type.fromOpcode(Message.java:102)
        at org.apache.cassandra.transport.Frame$Decoder.decode(Frame.java:179)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
        at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)

So is the DataStax client just passing up the exception from the server? Kind of weird, but plausible I guess.



--
You received this message because you are subscribed to the Google Groups "Spring Data Cassandra" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spring-data-cass...@googlegroups.com.
To post to this group, send email to spring-dat...@googlegroups.com.
Visit this group at http://groups.google.com/group/spring-data-cassandra.

For more options, visit https://groups.google.com/d/optout.

David Webb (Prowave)

unread,
Apr 11, 2014, 9:54:48 AM4/11/14
to spring-data-cassandra

Can you try using the DS Java Driver to connect directly without using SD C*?

 

I think we need to determine if that is the issue.  What version of the driver are you using? The latest is 1.0.6.

 

<dependency>

  <groupId>com.datastax.cassandra</groupId>

  <artifactId>cassandra-driver-core</artifactId>

  <version>1.0.6</version>

</dependency>

David van Geest

unread,
Apr 11, 2014, 11:03:56 AM4/11/14
to spring-data-cassandra
Sure. I'm using whatever driver spring-cql requires. According to SBT it's 1.0.5-dse:

....
[info]   +-org.springframework.data:spring-cql:1.0.0.M1
[info]   | +-com.datastax.cassandra:cassandra-driver-core:1.0.5-dse
....

I tried connecting with just the 1.0.6 driver. From watching wireshark, I can see that it gets a lot further. However, I still end up with an exception. Here's what I did:

scala> val cluster = Cluster.builder.addContactPoint("127.0.0.1").withCredentials("al2_dev", "password").build
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /127.0.0.1 (com.datastax.driver.core.TransportException: [/127.0.0.1] Unexpected exception triggered (java.lang.NoClassDefFoundError: Could not initialize class com.datastax.driver.core.Codec)))
  at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:185)
  at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:80)
  at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:812)
  at com.datastax.driver.core.Cluster$Manager.access$100(Cluster.java:736)
  at com.datastax.driver.core.Cluster.<init>(Cluster.java:92)
  at com.datastax.driver.core.Cluster$Builder.build(Cluster.java:705)
  at .<init>(<console>:33)
  at .<clinit>(<console>)
  at .<init>(<console>:7)
  at .<clinit>(<console>)
  at $print(<console>)

I verified that Cassandra was running and that I could still login with cqlsh.  The connection process with 1.0.6 gets to the point where the client sends the following CQL:

SELECT cluster_name, data_center, rack, tokens, partitioner FROM system.local WHERE key='local'

One interesting tidbit from Wireshark. Using the different drivers, the segment containing the user/pass is different.

Here is the TCP data for 1.0.5:

0040  d6 86 01 00 00 0f 00 00  00 15 00 00 00 11 00 61   ........ .......a
0050  6c 32 5f 64 65 76 00 70  61 73 73 77 6f 72 64      l2_dev.p assword 

Here is the TCP data for 1.0.6:

0040  d6 14 01 00 00 04 00 00  00 29 00 02 00 08 75 73   ........ .)....us
0050  65 72 6e 61 6d 65 00 07  61 6c 32 5f 64 65 76 00   ername.. al2_dev.
0060  08 70 61 73 73 77 6f 72  64 00 08 70 61 73 73 77   .passwor d..passw
0070  6f 72 64                                           ord              

Is there a good spec for the CQL binary protocol I could take a look at somewhere? I can send the wireshark captures if that would help.


For more options, visit https://groups.google.com/d/optout.

David Webb (Prowave)

unread,
Apr 15, 2014, 10:22:40 AM4/15/14
to spring-data-cassandra

David,

 

I think the issue with connecting to C* directly (without SDC*), indicates a potentially different issue with your C* setup, or possibly the combination of your C* instance and the driver.

 

I would post your question to the driver group (https://groups.google.com/a/lists.datastax.com/forum/#!forum/java-driver-user) and once you can connect with just the driver, let us know if you still cannot connect with SDC*.

 

Sorry to redirect you, but I don’t know what else to try.  The DS Driver guys are very knowledgeable and helpful.

 

Dave

David van Geest

unread,
Apr 15, 2014, 10:39:40 AM4/15/14
to spring-data-cassandra
David, 

Agreed. Thanks for your help!

-David



For more options, visit https://groups.google.com/d/optout.

David van Geest

unread,
Apr 15, 2014, 3:50:32 PM4/15/14
to spring-data-cassandra
OK, I'm back. Finally got a DataStax driver to connect to the cluster successfully.  Turns out I had some dependency conflicts (specifically commons-codec:commons-codec and com.google.guava:guava).

After playing around a bit and posting on the java-driver-user list, I've discovered the following:

- The DSE versions of the driver CANNOT authenticate to open-source C* 1.2, even if you use PlainTextAuthProvider. This was confirmed on the Datastax list: https://groups.google.com/a/lists.datastax.com/forum/#!topic/java-driver-user/Et2XYHI8xt0

- The non-DSE versions of the driver CAN authenticate to open-source C* 1.2. The use an entirely different mechanism (see the AuthInfoProvider class). I successfully got this to work outside of SDC.

- The 2.0.x versions of the driver, which do not seem to be split between DSE and non-DSE, CAN authenticate to open-source C* 1.2 (I tried it).

However, now I have a problem. I would like to continue using Spring Data Cassandra, but since it directly depends on the DSE driver, I have no way of authenticating to open-source C*. It seems that there are a couple solutions:

1. Spring Data Cassandra updates to use the 2.0.x driver. I understand this will take some work, but I'm guessing it's already planned for the project. Any timeframe on when this would be completed?

2. As a workaround, it seems like I could possibly force SDC to use the non-DSE driver. I would probably have to build the cluster bean myself, and then pass a reference of it to the session. Does this seem feasible?

-David


David Webb (Prowave)

unread,
Apr 15, 2014, 4:14:27 PM4/15/14
to spring-data-cassandra

Just exclude the dse driver from the SDC dependency, and then add your 1.0.6 non-dse driver as a dependency at the same level as SDC in your POM.  This might work with with the 2.0 driver but we haven’t tried it yet.  Thanks for the update to the group.  Your detailed explanation is sure to help the next guy. J

Reply all
Reply to author
Forward
0 new messages