Should we move CI from Travis-ci to Github Actions?

[Tatu Saloranta]

Looks like Travis-CI transition from .org to .com is now hitting us;
no CI build has succeeded for the past 15 days. While I did change
settings which should give us some free builds per month, I don't
think those 10,000 credits are enough for all Jackson components (not
even sure it'd cover jackson-databind).
While I understand that the business side of this may be necessary for
Travis the company, it means that either Jackson project would need to
pay up, or, perhaps, we should consider moving to something like
Github Actions.
As G.A seems to be picking momentum, that seems like a reasonable
idea, but it is a new system for me and I would need some help.

On migration: Jackson builds are rather simple, and aside from
optimization aspects (if there are ways to cache Maven deps, f.ex),
the only advanced part in Travis was the automatic publishing of
SNAPSHOT versions. And that is/was tricky just due to auth tokens. So
perhaps migration would not be horribly complicated.

Also... this might make it easier to consider dependency builds: so
that, for example, build of `jackson-core` (of certain branch) could
trigger cascading build of its dependencies (`jackson-databind`, most
modules).
Even if this was not fully automatic -- that is, we'd need to do some
static configuration -- it could be useful in exposing issues that
currently are not immediately apparent.
Or, possibly we could simply force daily/weekly rebuilds of a set of
repos (base modules, 3 dataformat repos) which should also catch
cross-version compatibility issues.

-+ Tatu +-

[Michel Krämer]

Hi Tatu,

I think this is a good idea. Please note that GitHub also limits the number of GitHub Actions minutes per month, depending on which plan you have. I can see that you are on the Pro plan so you should have 3000 minutes per month (the default for the free plan is 2000) [1]. There are other limits regarding the number of concurrent jobs etc. [2]. The limits should be more than enough for almost all open source projects -- I just wanted to mention them in case you have different requirements.

By the way, I'm using GitHub Actions for all my open source projects (including bson4jackson). I was an early adopter and have quite some experience with it. If you need help, just let me know.

Cheers,
Michel

[1] https://docs.github.com/en/get-started/learning-about-github/githubs-products
[2] https://docs.github.com/en/actions/reference/usage-limits-billing-and-administration

> --
> You received this message because you are subscribed to the Google Groups "jackson-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jackson-dev...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-dev/CAL4a10jimXsSZxHC80bwOEfnZjFS8PN6OMWwnRXvdpekehqaRw%40mail.gmail.com.

[Michel Krämer]

Oh, maybe I should mention that the 2000 (or 3000) minutes only apply to private repositories if I'm not mistaken.

Cheers,
Michel

[drewgs...@gmail.com]

I found this extensive-looking repository describing how to setup GitHub Actions to publish Maven builds https://github.com/jonashackt/github-actions-release-maven

-Drew

[Artem Smotrakov]

I think GitHub Action works pretty well. In addition, there are many useful actions that may be helpful. In particular, GitHub Actions can run CodeQL scans (static analysis checks), and report findings (including security ones) right in a pull request. I can add a config for CodeQL if you find it useful.

Artem

[Tatu Saloranta]

On Wed, Jun 23, 2021 at 10:51 PM 'Michel Krämer' via jackson-dev
<jacks...@googlegroups.com> wrote:
>
> Hi Tatu,
>
> I think this is a good idea. Please note that GitHub also limits the number of GitHub Actions minutes per month, depending on which plan you have. I can see that you are on the Pro plan so you should have 3000 minutes per month (the default for the free plan is 2000) [1]. There are other limits regarding the number of concurrent jobs etc. [2]. The limits should be more than enough for almost all open source projects -- I just wanted to mention them in case you have different requirements.

Thanks! Yes, makes sense; due to Bitcoin mining schemes, all CI/CD
providers are imposing bit stricter limits.

> By the way, I'm using GitHub Actions for all my open source projects (including bson4jackson). I was an early adopter and have quite some experience with it. If you need help, just let me know.

Thank you! This should be useful. I found this which also looks useful:

https://www.marcd.dev/articles/2021-03/mvncentral-publish-github

but specific repos to have a look at are even more useful.

Beyond basic set up of 2 workflows (I think):

1. Regular branch build over active branches, maybe PRs (depending on
limits); just do `mvn clean verify`, with couple of diff JDKs, can use
full caching (but has to refresh snapshot deps)
2. Snapshot publish from active branches, using the official JDK
(lowest one we can use, currently JDK 8)

(why 2 workflows? I assume one could create unified one for both but I
suspect separate ones are easier to do, even if there's slight
duplication)

What would be really cool tho would be to have either:

1. Dependency snapshots: from static setup of dependencies, publish of
`jackson-databind` for given branch would also trigger builds of (some
of) the dependencies (same branch): at least, say,
`jackson-modules-base` and all 3 dataformats repos
2. or, if too complex/inefficient, Cron/Daily builds of a subset of
all components -- similar to (1) wrt resolving dependencies, just
longer latency

the main goal of which would be to try to catch occasional regressions
where I (usually :) ) break some aspect of `jackson-databind` for
developing branch, but not noticing failure until building that
component.
On Travis world, Scala module has such a setup.

-+ Tatu +-

> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-dev/4BB4B52B-1B89-4B51-B2E1-877EC044F532%40googlemail.com.

[Tatu Saloranta]

On Thu, Jun 24, 2021 at 5:27 AM Artem Smotrakov
<artem.s...@gmail.com> wrote:
>
> I think GitHub Action works pretty well. In addition, there are many useful actions that may be helpful. In particular, GitHub Actions can run CodeQL scans (static analysis checks), and report findings (including security ones) right in a pull request. I can add a config for CodeQL if you find it useful.

I think CodeQL could be useful, my one concern with static analysis
tools is that sometimes there is noise and over time managing
overrides can become a bigger task. But availability of reports to
check every now and then is useful, especially if the false positive
rate is low.
Somewhat related, Jackson-databind has the settings for old LGTM
analysis (lgtm.yml), and that was useful in finding couple of things
to fix
(but also had warnings to disable).

So I'd be happy to get such a config, starting with maybe one project
(jackson-core or jackson-databind f.ex) first?

-+ Tatu +-

> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-dev/CANu7eQEOxsXq1oXwT7HrkWnecL%2BoS%3DNx%3De%3Db%3DS2rVXmue4YMjQ%40mail.gmail.com.

[Artem Smotrakov]

> So I'd be happy to get such a config, starting with maybe one project

> (jackson-core or jackson-databind f.ex) first?

Okay, let's give it a try then. I'll submit a pull request for jackson-core or jackson-databind, and we'll see how it goes.

In fact, LGTM is still alive, and yes it is based on CodeQL. I think LGTM provides a useful UI, metrics, etc, so I think it makes sense to keep the LGTM config despite CodeQL workflows.

Artem

To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-dev/CAL4a10jZp7NJNBD%2BmjhzhuegXgxpBKw96kN-t3uy0wmtO0j%3Drg%40mail.gmail.com.

[Tatu Saloranta]

On Fri, Jun 25, 2021 at 5:17 AM Artem Smotrakov <artem.s...@gmail.com> wrote:

> So I'd be happy to get such a config, starting with maybe one project

> (jackson-core or jackson-databind f.ex) first?

Okay, let's give it a try then. I'll submit a pull request for jackson-core or jackson-databind, and we'll see how it goes.

Cool

 

In fact, LGTM is still alive, and yes it is based on CodeQL. I think LGTM provides a useful UI, metrics, etc, so I think it makes sense to keep the LGTM config despite CodeQL workflows.

Yeah, it has the usual challenge of issues remaining being noise, but it did expose a few legit issues so it seems useful.

-+ Tatu +-

 

To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-dev/CANu7eQGKQhQS8ZYD%2BHZtWH16ZbjJjMmtXDzzEq_cc_QiVq94rQ%40mail.gmail.com.