How to change granularity at ingest time

[Lukáš Havrlant]

Hi,

we use Tranquility and we have one minute query granularity and one hour segment granularity. We wanna change it to one hour query granularity and one day segment granularity. How should we do it without losing (or maybe multiplying) our data during ingest time? What are the best practices? I know I can always reindex the old data but I would like to avoid data inconsistency during ingest time. 

Thank you.

Lukáš

[Fangjin Yang]

Tranquility creates tasks for a defined period of time and Druid segments can be different rollups and segments, so this should be fine. You should be able to update the config as the next time Tranquility runs a new set of tasks, they will have a new query and segment granularity.

[Lukáš Havrlant]

OK, I'll try. Thank you! 

--
You received this message because you are subscribed to a topic in the Google Groups "Druid User" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/druid-user/o_TT7twiyMM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to druid-user+...@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/0c74660c-6630-4dfb-b436-a01ad8990d82%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Lukáš Havrlant]

Hi Fangjin,

I tried on our test server and it didn't work. We already had a segment for interval 2015-11-30T07:00:00.000Z--2015-11-30T08:00:00.000Z and when I changed it to day segment granularity it failed to create a new task because of:

2015-11-30T09:06:52,825 INFO [ClusteredBeam-ZkFuturePool-8d6d80e1-f08d-4927-9a54-2ee63fd24ae0] com.metamx.tranquility.beam.ClusteredBeam - Global latestCloseTime[2015-11-30T07:00:00.000Z] for identifier[druid:prod:overlord/sdgadserver-videoimpression] has moved past timestamp[2015-11-30T00:00:00.000Z], not creating merged beam

2015-11-30T09:06:52,838 INFO [ClusteredBeam-ZkFuturePool-8d6d80e1-f08d-4927-9a54-2ee63fd24ae0] com.metamx.tranquility.beam.ClusteredBeam - Turns out we decided not to actually make beams for identifier[druid:prod:overlord/sdgadserver-videoimpression] timestamp[2015-11-30T00:00:00.000Z]. Returning None.

We use Druid 7.3 and Tranquility 0.4.2. What did I do wrong? 

[Fangjin Yang]

Hmm, I'll ping Gian and ask him to take a look at this issue.

[pja...@yahoo-inc.com]

Hello,

Changing the segment granularity without loosing data is something tranquility does not handle very well. I have done a PR to fix this - https://github.com/druid-io/tranquility/pull/52
However, this PR does require some change in RealtimePlumber Druid code to support creation of variable size segments - so the PR is on hold till that time.

Thanks,
Parag

[Gian Merlino]

Hey Lukáš,

Parag is right, tranquility does not deal well with changing segment granularities at this time. Right now the simplest way to change that is to stop ingestion for a bit, wait for the current set of tasks to fully complete, and then re-start ingestion.

Gian

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.

To post to this group, send email to druid...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/c875238e-d3af-4988-8969-d85da503e913%40googlegroups.com.

[Lukáš Havrlant]

Hi Gian, I don't think that will work or I don't fully understand your answer. Suppose I have segment granularity set to HOUR and I want to change it to DAY and I'll do it at 8:00. It'll fail because of the error I posted in the first message -- it won't create a new segment for the day because there are already some segments for this day. I think the correct way is: Change the config at 23:30 and restart Tranquility app. Tranquility won't create new tasks because there would still be old tasks for 23:00--23:59. Thus at this time nothing happens and the ingestion will continue without aby problem with the old configuration. Next hour (= next day) Tranquility will create a new set of tasks and this time with DAY granularity. I did it this way and it worked. 

[Gian Merlino]

Hey Lukáš,

Ah ha, that method actually works better than what I was thinking of! :)

I was thinking wait for the old tasks to finish and start up a new config that wouldn't conflict. If you were changing from HOUR to DAY then you'd need to do that on the next day. But changing it at 23:30 (assuming your windowPeriod is <30m) works even better.

Gian

On Tue, Dec 8, 2015 at 11:52 AM, Lukáš Havrlant <lu...@havrlant.cz> wrote:

Hi Gian, I don't think that will work or I don't fully understand your answer. Suppose I have segment granularity set to HOUR and I want to change it to DAY and I'll do it at 8:00. It'll fail because of the error I posted in the first message -- it won't create a new segment for the day because there are already some segments for this day. I think the correct way is: Change the config at 23:30 and restart Tranquility app. Tranquility won't create new tasks because there would still be old tasks for 23:00--23:59. Thus at this time nothing happens and the ingestion will continue without aby problem with the old configuration. Next hour (= next day) Tranquility will create a new set of tasks and this time with DAY granularity. I did it this way and it worked. 

--

You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/CAGTt3W9Low%3DvBwTLnMhNCW4KGnvwKx_%3D_KUq-QkeYBb8AxJo3g%40mail.gmail.com.