Using Mysql for Druid metadata using docker-compose

[saurabh singh]

Hi Druid Users,

Has anyone been able to use MySql for Druid's metadata using docker-compose file please? I have spent many hours trying to get this to work but I still get errors in broker component when configure mysql in my docker-compose.

Below are the errors I see in 'broker' and PFA my docker-compose.xml file with enrironment file which is being referred. Would appreciate your help. (Postgres works fine for me btw).

2021-02-23T15:15:22,464 DEBUG [qtp575779772-162] org.apache.druid.jetty.RequestLog - 172.16.1.16 POST //172.16.1.13:8082/druid/v2/sql HTTP/1.1
2021-02-23T15:15:23,063 ERROR [sql[402982ef-7105-490a-83d5-8528580c36d3]] org.apache.druid.curator.discovery.ServerDiscoverySelector - No server instance found for [druid/overlord]
2021-02-23T15:15:23,063 ERROR [sql[18a217cf-13a1-444b-99b2-726a55152af1]] org.apache.druid.curator.discovery.ServerDiscoverySelector - No server instance found for [druid/coordinator]
2021-02-23T15:15:23,063 ERROR [sql[2e472014-8050-4886-81e4-4583fa601b40]] org.apache.druid.curator.discovery.ServerDiscoverySelector - No server instance found for [druid/coordinator]
2021-02-23T15:15:23,063 ERROR [sql[41ce0e01-9e49-40ae-a7df-bc01a1c0b11c]] org.apache.druid.curator.discovery.ServerDiscoverySelector - No server instance found for [druid/overlord]
2021-02-23T15:15:23,072 INFO [NodeRoleWatcher[HISTORICAL]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeRoleWatcher - Node[http://172.16.1.14:8083] of role[historical] detected.
2021-02-23T15:15:23,073 INFO [NodeRoleWatcher[HISTORICAL]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeRoleWatcher - Node watcher of role[historical] is now initialized.
2021-02-23T15:15:23,064 WARN [sql[18a217cf-13a1-444b-99b2-726a55152af1]] org.apache.druid.sql.http.SqlResource - Failed to handle query: SqlQuery{query='SELECT datasource FROM sys.segments GROUP BY 1', resultFormat=OBJECT, header=false, context={}, parameters=[]}
java.lang.RuntimeException: org.apache.druid.java.util.common.IOE: No known server
        at org.apache.druid.sql.calcite.schema.SystemSchema.getThingsFromLeaderNode(SystemSchema.java:1008) ~[druid-sql-0.20.0.jar:0.20.0]

Thanks,

Saurabh

--

[Saurabh Singh]

Just to clarify the reason why I can’t use postgres is because my BI tool also has a postgres docker container and when I connect my BI tool to Druid’s network, it’s hitting Druid’s postgres instead of its own and failing.

So I thought the quickest would be to switch Druid to something other than postgres. So if someone’s has got Druid working with any db other than postgres, can you please suggest the docker compose file for that and the environment etc files? 

Thanks,

Saurabh

Sent from my iPhone

On 23 Feb 2021, at 15:22, saurabh singh <saura...@gmail.com> wrote:



<docker-compose.yml>

<environment>

[saurabh singh]

I got the mysql working with Druid now. PFA my docker-compose and environment file if anyone else wants to use mysql as Druid's meta data storage.

Thanks,

Saurabh

--

Saurabh Kumar Singh
+44 (0) 754 525 5396

[Peter Marshall]

Thank you Saurabh :). Would be cool if we could get this into docs somehow...