1.2.0 Beta Released

[Brian Hawkins]

https://github.com/kairosdb/kairosdb/releases

We still need to merge in some other branches, but the release is feature complete.

Brian

[zhang qiang]

Hi Brian, I've try this new release with Cassandra 3.11.0, I think there is a little problem, the default 'kairosdb.properties' file don't have "kairosdb.datastore.cassandra.write_buffer_job_queue_size" definition, when I start kairosdb use "kairosdb.bat run" the powershell give me error message like this:

18:20:20.862 [main] INFO  [TelnetServerModule.java:38] - Configuring module TelnetServerModule Exception in thread "main" com.google.inject.CreationException: Guice creation errors:

1) Explicit bindings are required and java.lang.Integer annotated with @com.google.inject.name.Named(value=kairosdb.datastore.cassandra.write_buffer_job_queue_size) is not explicitly bound.

  while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=kairosdb.datastore.cassandra.write_buffer_job_queue_size)

    for field at org.kairosdb.datastore.cassandra.CassandraConfiguration.m_writeBufferJobQueueSize(CassandraConfiguration.java:25)  at org.kairosdb.datastore.cassandra.CassandraModule.configure(CassandraModule.java:71)

1 error

        at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:435)

        at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:154)

        at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:106)

        at com.google.inject.Guice.createInjector(Guice.java:95)

        at com.google.inject.Guice.createInjector(Guice.java:72)

        at org.kairosdb.core.Main.<init>(Main.java:265)

        at org.kairosdb.core.Main.main(Main.java:311)

When I add line "kairosdb.datastore.cassandra.write_buffer_job_queue_size=10000" to the properties file, kairosdb can start normally, although I'm not sure what's the proper value should this property be.

[Brian Hawkins]

Ah crap, there is a bug, I'll post a new beta in a few hours.  Sorry about that.

Brian

[Brian Hawkins]

OK I've updated the release to beta 2 with new binaries.  All should be good to go.

Brian

[Kevin Burton]

"

• Moved from Thrift to CQL (increase write and read performance)"

Are you saying that the CQL version is actually faster than the Thrift version?

[Brian Hawkins]

It took a bit of tinkering but I got it to run faster.  One major factor is I'm able to determine where each data point needs to go so I can batch up data that goes to the same server.  Without the batching I would say it is about %5 faster.

Brian

[Kevin Li]

Last time you mentioned that query was significant faster, but writing part is slower. So you mean now even writing data is also faster than Thrift. That's great news. Can't wait to test it out.

Kevin

[Brian Hawkins]

It took a while to figure out how to make it go fast.  I was eventually able to make it go faster than thrift.  It could go faster still but there is something in the client library that is throttling it, I was talking to some guys at a conference and they found the same problem.  Supposedly it is part of some back pressure mechanism in the client.  Regardless it performs well.

Brian

[Riley Zimmerman]

Hi, I don't see kairosdb.datastore.cassandra.replication_factor in the kairosdb.properties for the CQL beta.  Has this variable changed?  

[Jean-Philippe Durot]

Hi,

Indeed, this variable is not used anymore. The CQL Cassandra schema created at startup has a replication factor of 1 and a replication class set to SimpleStrategy.

You have to manually update cassandra keyspace with cqlsh once created.

JP

[Riley Zimmerman]

Thanks.  I see this weird error when I modify the replication_factor to 3.  I'm running on a clean Cassandra 3.10 install, 3 nodes.  

cqlsh> ALTER KEYSPACE "metricdb" WITH REPLICATION = {'class': 'SimpleStrategy', 'replication_factor': '3'}  AND durable_writes = true;
ServerError: java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.cassandra.exceptions.ConfigurationException: Column family ID mismatch (found fccc4e30-7928-11e7-94ce-1583a8dd2182; expected fcb3bd20-7928-11e7-94ce-1583a8dd2182)

It seems to have worked anyway.  The describe schema has 3 and my 3 nodes own 100%.  Thinking it's okay but wanted to check if anyone had seen this or knew what it meant.  

cqlsh> DESCRIBE SCHEMA
CREATE KEYSPACE metricdb WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '3'}  AND durable_writes = true;
...

Are there any plans to add the replication count back to the kairosdb setup?  

[Jean-Philippe Durot]

Hello,

Didn't have this problem because by altering replication factor, we also switched from SimpleStrategy to NetworkTopologyStrategy.

Did you run "nodetool repair" on each node after altering keyspaces ?

[Riley Zimmerman]

Hi,

I kept SimpleStrategy since I'm testing this in just one datacenter, but I can try NetworkTopologyStrategy.  I didn't run a repair, but this was on a basically empty Cassandra.  I had just brought everything up and not run any workloads when I did the ALTER, and the repairs would have had to be done been after getting the ALTER error.  I haven't come across any issues yet, other than the error message, but will keep testing.  Thanks!

[Brian Hawkins]

I can add it back in.  My issue was some people thought they could change that value after the fact and it would change the replication in C* after the schema was already created.  I don't like configuration with that vague of a usage pattern.

Brian

[Riley Zimmerman]

It's no problem, I can modify it manually/automatically outside of KairosDB if in your experience that's safer for all.  Either way is fine with me.  

[Avind]

Not sure if this has already been answered in any other thread ..  Is there some date to have this release as stable/GA? 

[Riley Zimmerman]

Hi Brian,

Can kairosdb.datastore.cassandra.replication_factor be added back please?  I've been manually changing to 3, but it can get messy in my use case. I have a kafka consumer where if data is already there and waiting for the DB when I deploy a new install, that data gets sent out with replication 1 until I can change it.  Even if I don't have data waiting and can make the manual change before data comes in, I've sometimes seen data consistency issues that take a few repairs to resolve.  I could probably throw in some code to my own deployment stuff, but I'm thinking that others may want to be able to change it (assuming they understand what replication is doing like you pointed out).

Thanks!

[Brian Hawkins]

Done, it is in the develop branch.

You can specify the entire replication string that is passed to CQL.

Brian

[Riley Zimmerman]

Perfect, thank you Brian!  That is much better, both the description and the ability to set a more detailed replication strategy!  

[Sebastian Spiekermann]

Hi Brian,

your last commit (replication in .properties file) reminded me of something I meant to ask a few weeks ago but forgot:

When installing the Beta via the provided rpm, the service is automatically started in the post-install process. I'll add the `rpm -qp --scripts` output below my post.

This behaviour is not optimal imho. There's a bunch of stuff I need to do after installing K*: setting folder/file permissions to a technical user, switching the init-script for a modified (mainly for pointing to a different configuration path) and so on...

Right now, after K* is installed and started unintendedly, I have to stop it again, correct all folders and files to be owned by an other user than root (the log-file created a root-user is a pitfall, really!)... 

So how about removing the /etc/init.d/kairosdb start from the rpm postinstall step? I think the auto-start is nothing anybody would miss badly... Also there's no mention of this in the docs (no mention of the rpm possibility at all, btw), so I'd keenly vote for removing it!

Regards,

Sebastian

$ rpm -qp --scripts kairosdb-1.2.0-0.3beta.rpm

postinstall scriptlet (using /bin/bash):

#!/bin/bash

if ! type chkconfig &> /dev/null; then

update-rc.d kairosdb defaults

else

chkconfig --add kairosdb

chkconfig kairosdb on

fi

/etc/init.d/kairosdb start

[Brian Hawkins]

I'm ok with that change, I don't hear anyone else complaining.  I can see if you are using configuration management tool like puppet or chef it would be nice to not have it startup before being able to change the configs and folders.

Brian

[Sebastian Spiekermann]

Couldn't agree more, Brian. Thanks for removing it!

And btw: thanks also for accepting my PR #411. So happy to be on the upstream code from now on! :)

Regards,

Sebastian

[Madhukar Thota]

is there a GA version release soon or one more beta version? We are waiting for GA version to go into production.

On Monday, July 10, 2017 at 11:37:12 AM UTC-4, Brian Hawkins wrote:

[Brian Hawkins]

I'm glad you asked.  I just posted it this morning.  This is now your official notice :)  I still need to put some comments on the release and update documentation, just as soon as I finish putting out fires at work.

Brian