Sharing code between multiple jobs

[Sid Wiesner]

We have a few related jobs that have similar code, and in a couple places, that section of the code is duplicated between jobs.

Has anyone found a good way to share code between jobs without copy/paste? Ideally this was using some type of import, but open to any/all ideas.

Best,

Sid

[Aleksa Krolls]

This is an interesting idea! It would be cool if you could implement code from a reference library of job snippets (that was either a public OpenFn library, or pulled from an organization's own repo). We recently have been brainstorming how to best build a public job library to share OpenFn configuration examples with the community... I wonder if we could set it up in a way where users can implement the library in any new job code, eliminating the need to copy and paste. @Taylor, any ideas here? 

[Taylor Downs]

There are two available "solutions" here, but I'm not sure that either one is great. Will need to think about this some more and I'm intrigued by the idea of making either all job code available within a project space during runtime, or (maybe even cooler) allowing users to specify extensions at the project level (like organization-specific language-package additions) which get injected into each VM at runtime along with the expression, the language-package, and state. This would solve both the issue of wanting to "import" shared job code and it would allow for project-level one-to-one mapping tables (sometimes called "dictionaries").

For now: one way would be to collapse multiple jobs into one, define some custom functions up at the top, and then use them throughout—controlling which operations get executed based on the data. This "monolith" style job may be harder to maintain.

The other way would be applicable if the "shared code" were generic enough to qualify as a language-package addition. All the language-packages are open source, and if you find yourself doing the same thing over and over (searching for a record in SF, doing X if found and doing Y if not found, for example) you could write that code as a new named function in language-salesforce. Then, instead of duplicating that code across all the jobs you could call searchAndOperate(uuid_field, uuid_value, codeIfFound, codeIfNotFound) as an operation (just like "create" or "upsert") in language-salesforce.

I'm terrible at naming things, but I hope you get the idea!

--
You received this message because you are subscribed to the Google Groups "OpenFn Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openfn+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openfn/c454629f-e290-4986-b666-46ae903608bc%40googlegroups.com.

--

Taylor Downs | Head of Product

Open Function Group | How Data Integration Makes ICT4D Work