equivalent of RDruid of pydruid for Java

[Maxime Nay]

Hi,

From what I understand, there is no equivalent of pydruid or RDruid for Java. The only way to query druid in java is to use some code in the druid-server module (which is not super convenient)
Do you think it would be interesting to have an equivalent of pydruid for Java (ie: a basic translation of pydruid in Java)?

Thanks

[Fangjin Yang]

Hi Maxime,

I think I forgot to respond to your last question about this :P

Druid accepts query requests as POST requests, so any client that can POST can query Druid. In general, all Druid requests are RESTful. Although there is no java query client, it is very possible to write one using any http client. 

-- FJ

[Maxime Nay]

Hi Fangjin,

Well, I deleted the last question anyway since I found the answer by myself a few minutes after.

My current question is more like: would it be useful if I was translating pydruid in java, or is someone already doing this/or is there a better way to do it than by using http requests?

Maxime

[Fangjin Yang]

Hi Maxime,

You can look at the Druids builder class for queries. RDruid and Pydruid both issue http requests to Druid. I wonder if you can share a bit more about your use case and why you are interested in a java query client?

[Maxime Nay]

Hi Fangjin,

We recently started to experiment Druid as a data store for our real time data pipeline (as a replacement for MySQL).
We would like to easily retrieve our data from both our ad server application, written in Java, (mainly for frequency capping, ad pacing and other ad serving related stuff) and from some scripts written in Groovy.

Also, we would like to be able to use some kind of SQL-like language to query Druid (so some other developers who might need the data won't have to learn Druid's query format). This could be included in the java query client.

If you want more detail about our use cases, don't hesitate to ask!

Maxime

[Fangjin Yang]

Hi Maxime,

There is a Druid SQL Runner in the code base that does simple SQL statements for Druid with antlr grammar. It is rudimentary right now but can be extended to do more. That code could use some love and would be a great point of contribution.

-- FJ

[Maxime Nay]

Oh, I was not aware of this SQL Runner.
I will try to get more familiar with this part of the code during the week.

Thanks!

[Manh Tran]

Hi Maxime,

I am new member. I also want to build  demo to use Query class follow SQLRunner.java. The statement query demo as below:

private static final String sql= "select count(*) from wikipedia where"

        + " timestamp between '2010-02-01' and '2015-02-14'"

        + " and (namespace = 'article')"

        + " and language in ( 'en', 'fr' ) "

        + " group by granularity(timestamp, 'day'), language";

CharStream stream = new ANTLRInputStream(sql);

DruidSQLLexer lexer = new DruidSQLLexer(stream);

TokenStream tokenStream = new CommonTokenStream(lexer);

DruidSQLParser parser = new DruidSQLParser(tokenStream);

lexer.removeErrorListeners();

parser.removeErrorListeners();

lexer.addErrorListener(ConsoleErrorListener.INSTANCE);

parser.addErrorListener(ConsoleErrorListener.INSTANCE);

After I run my propgram the result as below:

        

       timestamp language count(*)

       2014-04-08T00:00:00.000Z en 6340.0

       2014-04-08T00:00:00.000Z fr 382.0

I think we can use Http client, but I have a issue about fomart of the statement query because I do not see any document support, I also have search DruidSQLLexer.java and DruidSQLParser.java but not found 

Many thanks,

MT

[Fangjin Yang]

Hi guys, see inline.

On Monday, April 7, 2014 9:01:48 PM UTC-7, Manh Tran wrote:

Hi Maxime,

I am new member. I also want to build  demo to use Query class follow SQLRunner.java. The statement query demo as below:

private static final String sql= "select count(*) from wikipedia where"

        + " timestamp between '2010-02-01' and '2015-02-14'"

        + " and (namespace = 'article')"

        + " and language in ( 'en', 'fr' ) "

        + " group by granularity(timestamp, 'day'), language";

CharStream stream = new ANTLRInputStream(sql);

DruidSQLLexer lexer = new DruidSQLLexer(stream);

TokenStream tokenStream = new CommonTokenStream(lexer);

DruidSQLParser parser = new DruidSQLParser(tokenStream);

lexer.removeErrorListeners();

parser.removeErrorListeners();

lexer.addErrorListener(ConsoleErrorListener.INSTANCE);

parser.addErrorListener(ConsoleErrorListener.INSTANCE);

After I run my propgram the result as below:

        

       timestamp language count(*)

       2014-04-08T00:00:00.000Z en 6340.0

       2014-04-08T00:00:00.000Z fr 382.0

I think we can use Http client, but I have a issue about fomart of the statement query because I do not see any document support, I also have search DruidSQLLexer.java and DruidSQLParser.java but not found 

There isn't too much documentation for the SQL parser. It was contributed as a side project, but it can definitely be expanded and shown some love. Perhaps the author can address some of the questions about it here.

[Maxime Nay]

Those classes (DruidSQLLexer and DruidSQLParser) are generated by https://github.com/metamx/druid/blob/master/server/src/main/antlr4/io/druid/sql/antlr4/DruidSQL.g4

Fangjin: I am not sure to understand what you would like me to do though. If I improve this code, would you like me to create a distinct project, or to directly modify druid server? To me it seems kind of weird to include druid-server as a dependency just to issue some queries. (or maybe I missed something :D )

Maxime

On Monday, April 7, 2014 9:01:48 PM UTC-7, Manh Tran wrote:

[Xavier Léauté]

The DruidSQLLexer and DruidSQLParser classes are generated at compile time from the antlr grammar.

The SQL grammar is defined here https://github.com/metamx/druid/blob/master/server/src/main/antlr4/io/druid/sql/antlr4/DruidSQL.g4

Each grammar file produces a corresponding Lexer and Parser class, which implements the antlr4 Parser and Lexer classes:

http://www.antlr.org/api/Java/org/antlr/v4/runtime/Parser.html

http://www.antlr.org/api/Java/org/antlr/v4/runtime/Lexer.html

The antlr4 documentation available online is still a bit sparse, but you can buy their book or look at the example code.

--
You received this message because you are subscribed to the Google Groups "Druid Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.
To post to this group, send email to druid-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/b0e72fcb-8982-426d-9d9d-4cae966d1286%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.