Scalding Input Filepaths

[cvk]

Hi,

A really basic question from a newbie. I have a huge hierarchy of folders in HDFS. I want to process all the files below a certain level. Can someone advise me as to how should I provide the input?

I don't want to use MultipleWritableSequenceFiles and give  comma separated names for all of those hundreds of directories.

Thanks in advance. 

Regards,

cvk

[cvk]

Update: It does work when I use the * wildcard character. However it fails upon encountering _logs files. Any workarounds?

[Miguel Ping]

I had a similar problem, try to use wildcard/globbing to match the layout. If your files have an extension that's easier than matching the exact structure.

Just make sure you put the path inside double quotes

[cvk]

For others' reference, yes Wildcard regexes in the paths did the trick. Thanks Miguel