Correct way of adding new columns in ingestionSpec of Druid dataSource

27 views
Skip to first unread message

Saurabh Pande

unread,
May 18, 2023, 4:17:43 AM5/18/23
to Druid User
Hi folks,

We had a use-case of adding new columns to our druid dataSource and as per druid docs, druid provides flexibility to add columns to dataSource on the go as schema is maintained at segment level.

We are just confused now how should we add new columns, there two ways as below,

1. Should we directly go on druid coordinator console, open ingestionSpec in data loader, add columns and submit ?
2. Should we first suspend the supervisor, update ingestionSpec as mentioned in above point and then run supervisor again?

We are looking forwards to what are the impacts in both approaches and what should be the recommended way.

Thank you!
Saurabh Pande

John Kowtko

unread,
May 19, 2023, 2:26:56 PM5/19/23
to Druid User
Hi Saurabh, 

You should be able to do option #1.   When you edit an existing spec and resubmit, the supervisor job should automatically suspend itself, pick up the new spec and resume operation.  

If you have any further questions or issues with this let us know.

Thanks.  John

Mrugesh Kadia

unread,
Jun 9, 2023, 2:36:37 AM6/9/23
to Druid User
Hi Saurabh,

I have used both methods, and both work fine.

Thanks,
Mrugesh Kadia
Reply all
Reply to author
Forward
0 new messages