1.2.0 Beta Released

449 views
Skip to first unread message

Brian Hawkins

unread,
Jul 10, 2017, 11:37:12 AM7/10/17
to KairosDB
https://github.com/kairosdb/kairosdb/releases

We still need to merge in some other branches, but the release is feature complete.

Brian

zhang qiang

unread,
Jul 11, 2017, 6:31:29 AM7/11/17
to KairosDB
Hi Brian, I've try this new release with Cassandra 3.11.0, I think there is a little problem, the default 'kairosdb.properties' file don't have "kairosdb.datastore.cassandra.write_buffer_job_queue_size" definition, when I start kairosdb use "kairosdb.bat run" the powershell give me error message like this:

18:20:20.862 [main] INFO  [TelnetServerModule.java:38] - Configuring module TelnetServerModule Exception in thread "main" com.google.inject.CreationException: Guice creation errors:
1) Explicit bindings are required and java.lang.Integer annotated with @com.google.inject.name.Named(value=kairosdb.datastore.cassandra.write_buffer_job_queue_size) is not explicitly bound.
  while locating java.lang.Integer annotated with @com.google.inject.name.Named(value=kairosdb.datastore.cassandra.write_buffer_job_queue_size)
    for field at org.kairosdb.datastore.cassandra.CassandraConfiguration.m_writeBufferJobQueueSize(CassandraConfiguration.java:25)  at org.kairosdb.datastore.cassandra.CassandraModule.configure(CassandraModule.java:71)
1 error
        at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:435)
        at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:154)
        at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:106)
        at com.google.inject.Guice.createInjector(Guice.java:95)
        at com.google.inject.Guice.createInjector(Guice.java:72)
        at org.kairosdb.core.Main.<init>(Main.java:265)
        at org.kairosdb.core.Main.main(Main.java:311)

When I add line "kairosdb.datastore.cassandra.write_buffer_job_queue_size=10000" to the properties file, kairosdb can start normally, although I'm not sure what's the proper value should this property be.

Brian Hawkins

unread,
Jul 11, 2017, 1:39:37 PM7/11/17
to KairosDB
Ah crap, there is a bug, I'll post a new beta in a few hours.  Sorry about that.

Brian

Brian Hawkins

unread,
Jul 12, 2017, 9:53:08 AM7/12/17
to KairosDB
OK I've updated the release to beta 2 with new binaries.  All should be good to go.

Brian

Kevin Burton

unread,
Jul 16, 2017, 3:44:37 PM7/16/17
to KairosDB
"
  • Moved from Thrift to CQL (increase write and read performance)"

Are you saying that the CQL version is actually faster than the Thrift version?

Brian Hawkins

unread,
Jul 17, 2017, 10:12:06 AM7/17/17
to KairosDB
It took a bit of tinkering but I got it to run faster.  One major factor is I'm able to determine where each data point needs to go so I can batch up data that goes to the same server.  Without the batching I would say it is about %5 faster.

Brian

Kevin Li

unread,
Jul 29, 2017, 12:04:27 AM7/29/17
to KairosDB
Last time you mentioned that query was significant faster, but writing part is slower. So you mean now even writing data is also faster than Thrift. That's great news. Can't wait to test it out.

Kevin

Brian Hawkins

unread,
Jul 31, 2017, 2:58:30 PM7/31/17
to KairosDB
It took a while to figure out how to make it go fast.  I was eventually able to make it go faster than thrift.  It could go faster still but there is something in the client library that is throttling it, I was talking to some guys at a conference and they found the same problem.  Supposedly it is part of some back pressure mechanism in the client.  Regardless it performs well.

Brian

Riley Zimmerman

unread,
Aug 2, 2017, 1:42:25 PM8/2/17
to KairosDB
Hi, I don't see kairosdb.datastore.cassandra.replication_factor in the kairosdb.properties for the CQL beta.  Has this variable changed?  

Jean-Philippe Durot

unread,
Aug 2, 2017, 5:18:43 PM8/2/17
to KairosDB
Hi,

Indeed, this variable is not used anymore. The CQL Cassandra schema created at startup has a replication factor of 1 and a replication class set to SimpleStrategy.
You have to manually update cassandra keyspace with cqlsh once created.

JP

Riley Zimmerman

unread,
Aug 4, 2017, 11:31:04 AM8/4/17
to KairosDB
Thanks.  I see this weird error when I modify the replication_factor to 3.  I'm running on a clean Cassandra 3.10 install, 3 nodes.  

cqlsh> ALTER KEYSPACE "metricdb" WITH REPLICATION = {'class': 'SimpleStrategy', 'replication_factor': '3'}  AND durable_writes = true;
ServerError: java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.cassandra.exceptions.ConfigurationException: Column family ID mismatch (found fccc4e30-7928-11e7-94ce-1583a8dd2182; expected fcb3bd20-7928-11e7-94ce-1583a8dd2182)

It seems to have worked anyway.  The describe schema has 3 and my 3 nodes own 100%.  Thinking it's okay but wanted to check if anyone had seen this or knew what it meant.  

cqlsh> DESCRIBE SCHEMA
CREATE KEYSPACE metricdb WITH replication
= {'class': 'SimpleStrategy', 'replication_factor': '3'}  AND durable_writes = true;
...

Are there any plans to add the replication count back to the kairosdb setup?  

Jean-Philippe Durot

unread,
Aug 7, 2017, 9:14:46 AM8/7/17
to KairosDB
Hello,

Didn't have this problem because by altering replication factor, we also switched from SimpleStrategy to NetworkTopologyStrategy.
Did you run "nodetool repair" on each node after altering keyspaces ?

Riley Zimmerman

unread,
Aug 7, 2017, 11:26:17 AM8/7/17
to KairosDB
Hi,

I kept SimpleStrategy since I'm testing this in just one datacenter, but I can try NetworkTopologyStrategy.  I didn't run a repair, but this was on a basically empty Cassandra.  I had just brought everything up and not run any workloads when I did the ALTER, and the repairs would have had to be done been after getting the ALTER error.  I haven't come across any issues yet, other than the error message, but will keep testing.  Thanks!

Brian Hawkins

unread,
Aug 10, 2017, 11:44:03 AM8/10/17
to KairosDB
I can add it back in.  My issue was some people thought they could change that value after the fact and it would change the replication in C* after the schema was already created.  I don't like configuration with that vague of a usage pattern.

Brian

Riley Zimmerman

unread,
Aug 10, 2017, 12:34:51 PM8/10/17
to KairosDB
It's no problem, I can modify it manually/automatically outside of KairosDB if in your experience that's safer for all.  Either way is fine with me.  

Avind

unread,
Oct 31, 2017, 1:38:49 AM10/31/17
to KairosDB
Not sure if this has already been answered in any other thread ..  Is there some date to have this release as stable/GA? 

Riley Zimmerman

unread,
Jan 11, 2018, 11:27:26 AM1/11/18
to KairosDB
Hi Brian,

Can kairosdb.datastore.cassandra.replication_factor be added back please?  I've been manually changing to 3, but it can get messy in my use case. I have a kafka consumer where if data is already there and waiting for the DB when I deploy a new install, that data gets sent out with replication 1 until I can change it.  Even if I don't have data waiting and can make the manual change before data comes in, I've sometimes seen data consistency issues that take a few repairs to resolve.  I could probably throw in some code to my own deployment stuff, but I'm thinking that others may want to be able to change it (assuming they understand what replication is doing like you pointed out).

Thanks!

Brian Hawkins

unread,
Jan 13, 2018, 10:10:59 PM1/13/18
to KairosDB
Done, it is in the develop branch.

You can specify the entire replication string that is passed to CQL.

Brian

Riley Zimmerman

unread,
Jan 15, 2018, 1:22:10 PM1/15/18
to KairosDB
Perfect, thank you Brian!  That is much better, both the description and the ability to set a more detailed replication strategy!  

Sebastian Spiekermann

unread,
Jan 16, 2018, 2:55:14 AM1/16/18
to KairosDB
Hi Brian,
your last commit (replication in .properties file) reminded me of something I meant to ask a few weeks ago but forgot:

When installing the Beta via the provided rpm, the service is automatically started in the post-install process. I'll add the `rpm -qp --scripts` output below my post.
This behaviour is not optimal imho. There's a bunch of stuff I need to do after installing K*: setting folder/file permissions to a technical user, switching the init-script for a modified (mainly for pointing to a different configuration path) and so on...
Right now, after K* is installed and started unintendedly, I have to stop it again, correct all folders and files to be owned by an other user than root (the log-file created a root-user is a pitfall, really!)... 

So how about removing the /etc/init.d/kairosdb start from the rpm postinstall step? I think the auto-start is nothing anybody would miss badly... Also there's no mention of this in the docs (no mention of the rpm possibility at all, btw), so I'd keenly vote for removing it!

Regards,
Sebastian

$ rpm -qp --scripts kairosdb-1.2.0-0.3beta.rpm

postinstall scriptlet (using /bin/bash):
#!/bin/bash

if ! type chkconfig &> /dev/null; then
update-rc.d kairosdb defaults
else
chkconfig --add kairosdb
chkconfig kairosdb on
fi

/etc/init.d/kairosdb start

Brian Hawkins

unread,
Jan 23, 2018, 9:14:37 PM1/23/18
to KairosDB
I'm ok with that change, I don't hear anyone else complaining.  I can see if you are using configuration management tool like puppet or chef it would be nice to not have it startup before being able to change the configs and folders.

Brian

Sebastian Spiekermann

unread,
Jan 26, 2018, 1:48:37 AM1/26/18
to KairosDB
Couldn't agree more, Brian. Thanks for removing it!
And btw: thanks also for accepting my PR #411. So happy to be on the upstream code from now on! :)

Regards,
Sebastian

Madhukar Thota

unread,
Feb 1, 2018, 11:46:14 AM2/1/18
to KairosDB
is there a GA version release soon or one more beta version? We are waiting for GA version to go into production.

On Monday, July 10, 2017 at 11:37:12 AM UTC-4, Brian Hawkins wrote:

Brian Hawkins

unread,
Feb 1, 2018, 12:03:47 PM2/1/18
to KairosDB
I'm glad you asked.  I just posted it this morning.  This is now your official notice :)  I still need to put some comments on the release and update documentation, just as soon as I finish putting out fires at work.

Brian
Reply all
Reply to author
Forward
0 new messages