Safely clean up mysql database

[abhish...@media.net]

Hi

As we use mysql for storing all the metadata for druid services and utilities like lookups, ingestion task details, segment info etc.

I have been using druid from quite some time and thus my table size( druid_audit, druid_tasks, druid_tasklogs etc) has increased a lot and need to clean older data(ingestion task details, obsolete lookup info etc). Is there any way to safely cleanup data so that none of the live services are affected.

Thanks

Abhishek Jain

[Gian Merlino]

Hi Abhishek,

In general it's safe to delete entries from audit and tasklogs tables whenever you want (well, safe so long as you have no need for the audit info).

It should also be safe to delete entries from the tasks table if they are not running (based on status_payload) and old enough (based on created_date) that nothing is still trying to read their status. A week should be more than safe.

Gian

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+unsubscribe@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/162a7b74-74fe-47f2-8132-f4d40a779e94%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Laxmikant Pandhare]

I wanted to clean druid metadata entirely because it is now pointing to another part after migration.

What are the tables I have to clean completely and start fresh druid services?

On Thursday, June 7, 2018 at 8:57:29 PM UTC-7 Druid User wrote:

Hi Abhishek,

In general it's safe to delete entries from audit and tasklogs tables whenever you want (well, safe so long as you have no need for the audit info).

It should also be safe to delete entries from the tasks table if they are not running (based on status_payload) and old enough (based on created_date) that nothing is still trying to read their status. A week should be more than safe.

Gian

On Wed, Jun 6, 2018 at 4:28 AM, <abhish...@media.net> wrote:

Hi

As we use mysql for storing all the metadata for druid services and utilities like lookups, ingestion task details, segment info etc.

I have been using druid from quite some time and thus my table size( druid_audit, druid_tasks, druid_tasklogs etc) has increased a lot and need to clean older data(ingestion task details, obsolete lookup info etc). Is there any way to safely cleanup data so that none of the live services are affected.

Thanks

Abhishek Jain

--
You received this message because you are subscribed to the Google Groups "Druid User" group.

To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.

[Ben Krug]

There's a list of tables here - https://druid.apache.org/docs/latest/dependencies/metadata-storage/#metadata-storage-tables

Usually it's all the tables in the 'druid' schema in MySQL.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/2b4d608e-9456-445a-a15b-a6b2ec921d06n%40googlegroups.com.

[Laxmikant Pandhare]

Yes, I cleared almost every database mentioned below.

•  druid_audit           
•  druid_config          
•  druid_dataSource      
•  druid_pendingSegments 
•  druid_rules           
•  druid_segments        
•  druid_supervisors     
•  druid_tasklocks       
•  druid_tasklogs        
•  druid_tasks  

thank you for your response.