How to use FlowFile attribute as target table column value

[MikeR]

Hi!

I have a file that i want to ingest that comes daily with different name my_file_YYYYMMDD

I would like to create a column ASOF_DATE in target table that uses value from the file name.

What is the best way to do that?

I have tried below but it did not work for me:

I added a property ASOF_DATE with value ${filename:substring( ${filename:length():minus(8)} )} on table definition screen - that did not work.

I added a property to Initialize Feed Parameters processor called ASOF_DATE with value ${filename:substring( ${filename:length():minus(8)} )} - that worked and i can see that is is correctly set in nifi data provenance.

I then added a column in table definition called ASOF_DATE type string, added Standardizer Default Value with value ${ASOF_DATE} - that does not work and column comes out blank.

I am using kylo 0.9.1, nifi-1.6.0-RC3 and standard-ingest template from https://github.com/Teradata/kylo/blob/master/samples/templates/nifi-1.0/data_ingest.zip

Thank you.

[Greg Hart]

Hi Mike,

The Data Ingest template doesn't support adding a column with the filename. You will need to create your own template to accomplish this. Please see the 'Designer Tutorials' on kylo.io to help you get started:

https://kylo.io/tutorials.html

You may also be interested in contacting Think Big Analytics for a services engagement to create any templates you might need.