Flat file based sink

[sridh...@gmail.com]

Hi,

Is it possible to use a unix file system based sink in production env?

From the "getting started" documentation I know by default divolte writes avro files to /tmp. Is it possible to define local filesystem based sink with some unix path in configuration?

We wanted to use divolte collector for click stream data tracking for our website that currently do not have too much of traffic. But anticipate in future.

So we like the ability to use kafka and HDFS which I am sure we will use in future, but currently we are not ready with the infrastructure for kafka/HDFS yet.

We want to start with file (unix) based sink currently and later move to kafka and HDFS later. Please let me know if it is possible?

Thanks,

Sridhar

[Friso van Vollenhoven]

Hi Sridhar,

Through the HDFS sink, it is actually possible to write to the local file system, the same way the examples do that. If you configure it like that, there is no option to also write to HDFS from the same instance, as HDFS configuration for the Hadoop clients are classpath bound.

Cheers,

Friso

--
You received this message because you are subscribed to the Google Groups "divolte-collector" group.
To unsubscribe from this group and stop receiving emails from it, send an email to divolte-collec...@googlegroups.com.
To post to this group, send email to divolte-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/divolte-collector/0c7d84f1-bf48-40ea-bcc3-8488e84d1e99%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.