How can I feed last successful run timestamp into next job ?

Vinod Shankar K

unread,

Feb 15, 2024, 11:53:46 AMFeb 15

to CDAP User

Solutions Larroude

unread,

Feb 19, 2024, 2:26:54 PMFeb 19

to cdap...@googlegroups.com

Hello,

I am extracting data from a postgre database through VPN for CDAP data fusion.

I have some pipelines that are working correctly, but only one has the following error below:

Spark 'phase 1' program failed.
Spark program 'phase-1' failed with error: canCommit() is called for transaction {transaction number} that is not in progress (is known to be invalid). Check system logs for more details.
Pipeline 'pipeline name' failed.
The workflow service 'workflow.{client project name}.{pipeline name}.DataPipelineWorkflow.253bbfb9-cf50-11ee-a96a-0000007dd528' failed.

Can someone help me?
Because the Data Fusion log does not show the necessary information about this error.

The information between { } is customer data, so I did not make it available.

On Thu, Feb 15, 2024 at 1:53 PM 'Vinod Shankar K' via CDAP User <cdap...@googlegroups.com> wrote:

How can I feed last successful run timestamp into next job ?

--
You received this message because you are subscribed to the Google Groups "CDAP User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdap-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/fcf65dd0-0956-4568-a04e-7c29e744a8acn%40googlegroups.com.

Solutions Larroude

unread,

Mar 15, 2024, 5:42:05 PMMar 15

to cdap...@googlegroups.com

Sorry.

This request was resolved.

Thank

On Thu, Feb 15, 2024 at 1:53 PM 'Vinod Shankar K' via CDAP User <cdap...@googlegroups.com> wrote: