Rhino ETL - run operations when whole process succeeded

[Filip Kinsky]

I'm thinking about right Rhino ETL process/operation design for my current scenario, which includes file operations and transactional DB access. SImply it's an import process which should read some files, import data from the files into DB in transactional manner and if all the DB operations succeed than it should perform some additional file operations.

I could split the import process in following operations:

1. look for files to process in some directory (produces N files as result)

2. parse each file (produces a Row for each record in each file)

3. insert each record into DB

--

4. create backup of each file found in operation 1

5. delete processed file found in operation 1

The problem is that operations 4+5 should be run only if all rows were successfully inserted into DB (transaction already commited). Is this any well-known Rhino-ETL scenario with some elegant solution? I could override EtlProcess.Execute to wrap operations 1-3 in transaction and run operations 4+5 manually (ie. not registering them into process using EtlProcess.Register(op), but running them explicitely) just in case operations 1-3 were completed without any errors and transaction was commited, but I don't like this solution much. I feel it like there's probably some better solution for this, but can't figure it out at the moment... Any ideas?

[Miles Waller]

Hi

I believe EtlProcess has a postProcessing method that is called once thepopeline has run. You should be able to plug your own implementation in there.

Miles

--
You received this message because you are subscribed to the Google Groups "Rhino Tools Dev" group.
To view this discussion on the web visit https://groups.google.com/d/msg/rhino-tools-dev/-/5-7Prq3S5JAJ.
To post to this group, send email to rhino-t...@googlegroups.com.
To unsubscribe from this group, send email to rhino-tools-d...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rhino-tools-dev?hl=en.

[Filip Kinsky]

You're probably right, thanks. I could override PostProcessing to perform operations 4+5 and I should probably create my own TransactionalPipelineExecuter to wrap oprations 1-3 into DB transaction.