How to write into S3 using fixed filename

Skip to first unread message

jayesh wasnik

Oct 12, 2021, 2:58:47 AM10/12/21
to sdc-user

I am using " Amazon S3" Destination to write some file into S3 everyday.
I am putting the same JSON file into dated folders,so everyday this file would be placed in the respective date folder.
But the name of file is different each day,as in the time basis field I have added ${time:now()} so this gets appended to the name of the file.

Can you please help me in removing this timestamp value,or to find a way to write the file with same name everyday into separate folder.

I have attached a screenshot of my configurations too.

jayesh wasnik

Oct 12, 2021, 3:01:13 AM10/12/21
to sdc-user, jayesh wasnik


Sujata hs

Feb 16, 2022, 9:58:41 AM2/16/22
to sdc-user,
Hey Jayesh,

You can use 2 pipeline to achieve this.

1. first write original pipeline data to LocalFS
2. second pipeline - Read from local FS & write to your actual destination as "wholeFileTransfer"
    You Can use the  required filename in "FileNameExpression" configuration [attaching the screenshot]

Reply all
Reply to author
0 new messages