Can s3 Connector target multiple level partitions

[jeff whybark]

Hello,

I am looking to potentially leverage the Confluent s3 Connector.  I would like to be able to write out to a multi-level partition in s3 based off of a FieldPartitioner and supply the partition.field.name.  These seems simple enough if I have a single column to partition on, but what if I want to partition on two or more columns.  I can see that the TimeBasedPartitioner can handle the multi-level partitions (path.format='year'=YYYY/'month'=MM/'day'=dd/'hour'=HH/), but what about FieldPartitioner?

Thanks,

Jeff

[Gwen Shapira]

Hey,

Currently FieldPartitioner only handles a single field. 

Looking at the code, adding multi-field support doesn't look very complex: 
https://github.com/confluentinc/kafka-connect-storage-common/blob/master/partitioner/src/main/java/io/confluent/connect/storage/partitioner/FieldPartitioner.java

Feel free to open a github issue or submit a PR :)

Gwen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/90d3bb4e-02d2-40df-a7ac-aff0b784edbd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.