Merging files

[Roger Hoover]

Hi all,

The Camus job produces a lot of files that over time will build up and use a lot of resources on the Namenode.  What's the best practice for merging these into larger files?

Thanks,

Roger

[Félix GV]

I think the camus_sweeper module in the Camus repo is there for this purpose, but I'm not sure.

I think it hasn't been announced officially yet, because they want to do some documentation for it first. Again, I'm not sure, but if you feel like figuring things out on your own, I'd recommend starting there.

--
Félix

--
You received this message because you are subscribed to the Google Groups "Camus - Kafka ETL for Hadoop" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camus_etl+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Roger Hoover]

Thanks!