Mismatch between tranquility and druid in specifying approximate histogram aggregtion type

Radhakrishnan Jankiraman

unread,

Feb 7, 2017, 1:04:55 AM2/7/17

to Druid User

If I specify an ingestion time aggregation as

{

"type": "approximateHistogram",

"name": "loadtime",

"fieldName": "loadtime"

}

Tranquility errors out. If I change "approximateHistogram" to "histogram" tranquility doesn't complain, but it causes an error in Druid

These are the versions of tranquility and druid

compile group: 'io.druid', name: 'druid-api', version: '0.9.2'

compile group: 'io.druid', name: 'tranquility-core_2.11', version: '0.8.2'

Any ideas ?

Thanks,

Radhakrishnan

Gian Merlino

unread,

Feb 7, 2017, 1:08:29 AM2/7/17

to druid...@googlegroups.com

The aggregator types are "approxHistogramFold" and "approxHistogram", not "approximateHistogram". For ingestion you should use "approxHistogramFold".

If that doesn't help, you also might need to load the extension into Tranquility, see: https://github.com/druid-io/tranquility/blob/master/docs/configuration.md#loading-druid-extensions

Gian

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+unsubscribe@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/fa94c4eb-04b3-4eed-8532-84622dcbd767%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Radhakrishnan Jankiraman

unread,

Feb 7, 2017, 4:24:07 AM2/7/17

to Druid User

This

{

"type": "approxHistogram",

"name": "loadtime",

"fieldName": "loadtime"

},

{

"type": "approxHistogram",

"name": "connecttime",

"fieldName": "connecttime"

}

Leads to

Exception in thread "main" java.lang.IllegalArgumentException: Could not resolve type id 'approxHistogram' into a subtype of [simple type, class io.druid.query.aggregation.AggregatorFactory]: known type ids = [AggregatorFactory, cardinality, count, doubleMax, doubleMin, doubleSum, filtered, histogram, hyperUnique, javascript, longMax, longMin, longSum]

at [Source: N/A; line: -1, column: -1] (through reference chain: io.druid.segment.realtime.FireDepartment["dataSchema"]->io.druid.segment.indexing.DataSchema["metricsSpec"]->Object[][1])

at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3425)

at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:3351)

at com.metamx.tranquility.druid.DruidBeams$.makeFireDepartment(DruidBeams.scala:433)

at com.metamx.tranquility.druid.DruidBeams$.fromConfigInternal(DruidBeams.scala:299)

at com.metamx.tranquility.druid.DruidBeams$.fromConfig(DruidBeams.scala:204)

at com.metamx.tranquility.druid.DruidBeams$.fromConfig(DruidBeams.scala:123)

at com.metamx.tranquility.druid.DruidBeams.fromConfig(DruidBeams.scala)

at com.appdynamics.eum.resourcetiming.SamplingBasedDataIngester.main(SamplingBasedDataIngester.java:27)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

Caused by: com.fasterxml.jackson.databind.JsonMappingException: Could not resolve type id 'approxHistogram' into a subtype of [simple type, class io.druid.query.aggregation.AggregatorFactory]: known type ids = [AggregatorFactory, cardinality, count, doubleMax, doubleMin, doubleSum, filtered, histogram, hyperUnique, javascript, longMax, longMin, longSum]

at [Source: N/A; line: -1, column: -1] (through reference chain: io.druid.segment.realtime.FireDepartment["dataSchema"]->io.druid.segment.indexing.DataSchema["metricsSpec"]->Object[][1])

at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)

at com.fasterxml.jackson.databind.DeserializationContext.unknownTypeException(DeserializationContext.java:964)

at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._handleUnknownTypeId(TypeDeserializerBase.java:271)

at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:159)

at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:108)

at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:93)

at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:131)

at com.fasterxml.jackson.databind.deser.std.ObjectArrayDeserializer.deserialize(ObjectArrayDeserializer.java:158)

at com.fasterxml.jackson.databind.deser.std.ObjectArrayDeserializer.deserialize(ObjectArrayDeserializer.java:17)

at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:520)

at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:461)

at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:377)

at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1100)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:294)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:131)

at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:520)

at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:461)

at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:377)

at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1100)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:294)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:131)

at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3420)

... 12 more

Specifying the following JVM properties

-Ddruid.extensions.directory=/Users/jrkrishnan/druid-0.9.2/extensions -Ddruid.extensions.loadList='["druid-kafka-eight","druid-s3-extensions","druid-histogram","druid-datasketches","druid-lookups-cached-global","mysql-metadata-storage"]'

Leads to

Exception in thread "main" java.lang.ExceptionInInitializerError