Should we use Dependabot?

[Josh Matthews]

Hello all! You may have noticed some new pull requests from Dependabot
[1], which is a service to create automated pull requests to update
dependencies.

There are several things I like about this service:
* it provides links to changelogs/release notes/included commits which
makes reviewing easier
* it automatically rebases its PRs if merge conflicts appear
* it supports quite a bit of customization [2]
* it removes the need for a member of the team to regularly check for
new releases that fix security vulnerabilities

One concern that has been raised [3] is the tension between updating
packages and minimizing the set of duplicated packages. If Dependabot is
used in the default setting of updating every dependency to the latest
version as often as possible, we will likely introduce duplicate crates
more frequently than we currently do. Our existing test-tidy check for
duplicates makes this mode less useful for us, because it requires
modifying the relevant Dependabot PRs to update the list of allowed
duplicates when they are added.

Dependabot does support operating modes where certain packages are
exempt from automatic updates, or only making PRs for semver-minor
updates and ignoring major version changes. These modes might reduce how
often we need to update the list of duplicates, but it's still possible
for minor version changes to introduce new duplicates.

I have a few questions that I'm interested in hearing feedback on:
* should we use Dependabot at all?
* is our policy to ban duplicate versions by default still useful?
* what changes should we make to the policy to accommodate the use of
Dependabot?

Cheers,
Josh

ps. Before forming an opinion on the utility of Dependabot, please take
a look at the collapsed-by-default "Dependabot commands and options" [4]

[1] https://dependabot.com/, https://github.com/servo/servo/pull/26256
[2] https://dependabot.com/docs/config-file/
[3] https://github.com/servo/servo/issues/26262
[4] https://github.com/servo/servo/pull/26256#issue-407135420

[Simon Sapin]

On 22/04/2020 20:30, Josh Matthews wrote:
> * it provides links to changelogs/release notes/included commits which
> makes reviewing easier

Does this also work for crates.io dependencies? (As opposed to git dependencies.)
Does it rely on the upstream repository having git tags that happen to match the
crates.io version numbers?

--
Simon Sapin

[Josh Matthews]

Based on https://github.com/servo/servo/pull/26255 and
https://github.com/servo/servo/pull/26258 which are both crates.io
dependencies, it does seem like it relies on some metadata that is not
present in every dependency.

[Cheng You Bai]

I think it's quite great to have Dependabot for us to upgrade dependencies
for security purposes.

I'm wondering if it's worth keeping it enabled but sometimes reject its PRs
(e.g. having duplicated packages or the upgrade makes something broken
which will require someone's effort to fix).

I also wonder it might be good to enable it with no-checking for duplicated
packages for a while to see if we can get benefits from it.
If we get build performance worse, then let's get back the checking of
duplicated packages? 🤔

On Thu, Apr 23, 2020 at 3:58 AM Josh Matthews <jo...@joshmatthews.net> wrote:

> Based on https://github.com/servo/servo/pull/26255 and
> https://github.com/servo/servo/pull/26258 which are both crates.io
> dependencies, it does seem like it relies on some metadata that is not
> present in every dependency.
>
> On 2020-04-22 2:45 p.m., Simon Sapin wrote:

> _______________________________________________
> dev-servo mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-servo
>

[Anthony Ramine]

Hello,

> * should we use Dependabot at all?

Personally I don't like it. We most often know when we should be updating stuff, and making semver-compatible bumps for the sake of making them doesn't seem very important to me.

> * is our policy to ban duplicate versions by default still useful?

Yes, definitely.

> * what changes should we make to the policy to accommodate the use of Dependabot?

I would like it to make issues instead of PRs. Maybe in its own Github project or something like that. And only for semver-incompatible bumps.

[Bastien Orivel]

Hi,

> I have a few questions that I'm interested in hearing feedback on:

> * should we use Dependabot at all?

I personally don't think we should use Dependabot.

Looking at the current PRs it made, the `time` one I'm 99% sure needs
code changes and would introduce a duplicate. The `keyboard-types` one
is probably wrong, would introduce a dupe in a crate used for sharing
types across crates (would probably not compile). The `image` one would
dupe png. The `cc` and `smallvec` ones break the build. The `winit` one
doesn't build, would bring in more dupes.

> * is our policy to ban duplicate versions by default still useful?

Yes. Servo's dependency graph is huge already, let's not make it worse
by having 3 versions of the same dependency for every dependency.

> * what changes should we make to the policy to accommodate the use of
> Dependabot?

If it opened issues on semver breaking changes and maybe pinged people
that like updating dependencies the it might be better. Some of those
might even be good first issues like the time one if we can provide
examples of similar bumps.

Regards,
Bastien

[Alan Jeffrey]

The problem I'm having with dependabot is that it opens PRs for upgrades
that won't pass CI without a lot of work, e.g. upgrading winit (
https://github.com/servo/servo/pull/26256), and as a result I treat emails
I get for dependabot PRs as quite likely to involve wasted effort.

The situation would be much better if we could somehow get the emails to be
issued only if the PR passes the initial taskcluster build in CI. For
example, if dependabot opened a draft PR, and only made it a full PR if the
initial CI run succeeds? (And if we don't assign a reviewer to draft PRs.)

Alan.

[Josh Matthews]

In my experience, Taskcluster returns results within 30 minutes of the
PR opening, so all I need to do is check the in-PR results for a green
checkmark. If it's there, it's easy enough to merge. If it's red, it
sometimes means I need to file an issue like
https://github.com/servo/mozangle/issues/38 or
https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/issues/251; in
any case, dependabot PRs that require code changes or de-duplication can
be closed without incurring extra work.

Cheers,
Josh

[Anthony Ramine]

To me, the very act of having to acknowledge more email notifications is being incurred extra work.

> Le 4 mai 2020 à 18:53, Josh Matthews <jo...@joshmatthews.net> a écrit :
>
> In my experience, Taskcluster returns results within 30 minutes of the PR opening, so all I need to do is check the in-PR results for a green checkmark. If it's there, it's easy enough to merge. If it's red, it sometimes means I need to file an issue like https://github.com/servo/mozangle/issues/38 or https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/issues/251; in any case, dependabot PRs that require code changes or de-duplication can be closed without incurring extra work.
>
> Cheers,
> Josh
>
> On 2020-05-01 11:08 a.m., Alan Jeffrey wrote:

[Jan Andre Ikenmeyer]

You could block this user: https://github.com/dependabot-bot/

Am 20.05.20 um 10:44 schrieb Anthony Ramine:

[Jan Andre Ikenmeyer]

If that doesn't work anymore, you could add a filter rule to Gmail or
Thunderbird. (That's what I did.)

Am 20.05.20 um 10:53 schrieb Jan Andre Ikenmeyer via dev-servo: