The end instant must be greater or equal to the start

[art...@leantegra.com]

Hi

We have Druid v 0.8.3

So when we do a query to Druid we have an error “The end instant must be greater or equal to the start”.

The main points of such query are

1. Granularity with any origin monthly wrapped by 30 or 31th day of the month

"granularity": {
"type": "period",
"period": "P1M",
"origin": "2014-11-30T22:00:00.000Z"
}

2. Second the period should contain the March 30 (2015 is the only year we have data) e.g.

"intervals": [
        "2015-02-01T22:00:00.000Z/2015-04-30T20:59:00.000Z"
]

Eventually we are making conclusion that it is connected with daylight savings time on March 29 2015.

But anyway we are not sure what is defiantly wrong. And how exactly can we handle this case.

Thank you.

[Xavier Léauté]

Hi there, can you provide a full query that reproduces the problem and what timezones you are running your servers on?

Given that you don't provide a timezone, you should not be running into issues if UTC is being used everywhere.

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/68f2bd7e-0884-4ab3-ae03-bd0902296fa6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[art...@leantegra.com]

Hi. These are

-Duser.timezone=UTC

{
    "queryType": "topN",
    "dataSource": "visitors_passersby",
    "intervals": [
        "2014-11-30T22:00:00.000Z/2015-03-31T20:59:00.000Z"
    ],

    "granularity": {
        "type": "period",
        "period": "P1M",

        "timeZone": "+00:00",
        "origin": "2014-11-30T22:00:00.000Z"
    },
    "dimension": {
        "type": "default",
        "dimension": "loc_id"
    },
    "aggregations": [{
        "type": "filtered",
        "filter": {
            "type": "selector",
            "dimension": "is_visitor",
            "value": true
        },
        "aggregator": {
            "type": "hyperUnique",
            "name": "visitors",
            "fieldName": "uniq_id"
        }
    }, {
        "type": "hyperUnique",
        "name": "total_uniq",
        "fieldName": "uniq_id"
    }],
    "postAggregations": [{
        "type": "arithmetic",
        "name": "passersby",
        "fn": "-",
        "fields": [{
            "type": "hyperUniqueCardinality",
            "fieldName": "total_uniq"
        }, {
            "type": "hyperUniqueCardinality",
            "fieldName": "visitors"
        }]
    }],
    "metric": {
        "type": "inverted",
        "metric": "visitors"
    },
    "threshold": 1
}

[Gian Merlino]

Hey Artemk,

This looks like a problem with the PeriodGranularity origin of 2014-11-30T22 (and in general origins that are late in the month). Do you need an origin that late in the month? If not, a similar query should work with an origin earlier in the month.

See also: https://github.com/druid-io/druid/issues/2479

Gian

--

You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/159416ed-acea-4715-9f86-f5d392886b36%40googlegroups.com.

[art...@leantegra.com]

According to the requirements we need exactly strict monthly ranging. So an earlier origin seems not to be a good decision for us.

So is there any another better solution?

Thx!

[Gian Merlino]

What do you mean by 'exactly strict monthly ranging'? Perhaps there is some other way you could write the query.

Gian

--

You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/fd8382f3-405c-4e07-8155-fa6af3d319c3%40googlegroups.com.

[Trevor Jang]

{

  "queryType" : "groupBy",

  "dataSource" : "data",

  "granularity" : {

    "type" : "period",

    "period" : "P10Y",

    "timeZone" : "America/Los_Angeles",

    "origin" : "2016-03-01T00:25:11.484Z"

  },

  "filter" : {

    "type" : "and",

    "dimension" : null,

    "value" : null,

    "fields" : [ {

      "type" : "selector",

      "dimension" : "ckey",

      "value" : "17943e6c6eec49cdb6",

      "fields" : null,

      "field" : null

    }, {

      "type" : "selector",

      "dimension" : "type",

      "value" : "errmsg",

      "fields" : null,

      "field" : null

    } ],

    "field" : null

  },

  "aggregations" : [ {

    "type" : "count",

    "fieldName" : "errmsg",

    "name" : "err",

    "fieldNames" : null

  } ],

  "postAggregations" : null,

  "intervals" : [ "2016-03-01T00:25:11.484Z/2016-03-01T00:40:11.483Z" ],

  "dimensions" : [ ],

  "limitSpec" : null,

  "having" : null,

  "context" : null

} 

I used the query above and got same error message 'The end instant must be greater or equal to the start' back then, but it does not show error message anymore right now.

(The reason I used 'P10Y' is that I want apply timezone to 'all' granularity.)

To avoid those error, how to rewrite this query?

[Fangjin Yang]

Trevor, just an FYI, the query you are issuing is incredibly inefficient to just using a timeseries query. You may want to consider just using https://github.com/implydata/plyql

[Trevor Jang]

Oh, I have missed that point. Thanks for your heads up, Fangjin! :)