Versioning Exported Dependencies

[Natalie Weizenbaum]

Hello Dartisans,

I've recently come across a tricky edge case when it comes to versioning dependencies, in particular dependencies that are explicitly exported by the depender. For example, in the Dart repo, there are three packages that export other packages: unittest which exports matcher, scheduled_test which exports unittest, and polymer which exports observe. When one package exports another, the exported API is generally considered part of the exporter's API. Users' version constraints reflect this: many packages use the matcher APIs from unittest, but none of them depend on matcher directly.

The Problem

Thursday morning, unittest 0.11.1+1 and matcher 0.11.1+1 were the latest versions. unittest had the constraint matcher: ">=0.10.0 <0.12.0". Thursday afternoon, I committed a CL that added the isNotEmpty property to matcher and bumped its version to 0.11.2.

Suppose a user creates a new project that depends on unittest. To make sure other users get a version of unittest that has all the features they're using, they use the constraint unittest: ">=0.11.1 <0.12.0". They then proceed to use isNotEmpty.

Now we have a problem. According to the version constraints, using unittest 0.11.1+1 and matcher 0.11.1+1 is perfectly valid. However, in practice, doing so will break the package, since it uses a feature that's only available in matcher 0.11.2.

The Solution

We need to ensure that a lower bound on a dependency restricts the features it provides in practice. Concretely, if a user writes unittest: ">=0.11.2 <0.12.0" they should be confident that import "package:unittest/unittest.dart" will always provide a superset of the functionality of the lowest version they can test against when they write their code.

The way to do this is to narrow the constraint on exported dependencies substantially. In the example above, unittest 0.11.1+1 would have the constraint matcher: ">=0.11.1 <0.11.2". Then when matcher 0.11.2 was released, unittest 0.11.2 would be released as well with the constraint matcher: ">=0.11.2 <0.11.3".

The tight lower bound ensures that a user's constraint of unittest: ">=0.11.1 <0.12.0" translates into a lower bound of 0.11.1 on matcher as well. The tight upper bound ensures that when matcher 0.11.2 is released, users won't get it without also getting a new version of unittest, signaling that they should change their lower bound if they're using any new features.

This does put more work on the shoulders of the authors of the exporting packages, since they need to rev whenever the package they export adds a new feature. That said, it also makes a certain amount of sense: by exporting another package's API, they're tightly coupling themselves to that package, so it's reasonable for the version constraint to reflect that.

I've updated the Dart repo packages to follow the new convention, and soon we'll update the versioning guidelines to document it as well. I'll also send out an email to all authors of pub.dartlang.org packages that export other packages suggesting that they switch to the new convention.

Let me know if you have any questions!

- Natalie

[Anders Holmgren]

Are you planning to update dart publish command to warn in this case?

[Natalie Weizenbaum]

Currently the publish command doesn't have any infrastructure to examine the actual content of Dart files, including imports and exports. I've wanted to do stuff like that for a while with the analyzer package, but it's a considerable amount of work that I haven't had time to do yet.

On Tue, Dec 9, 2014 at 3:57 PM, Anders Holmgren <andersm...@gmail.com> wrote:

Are you planning to update dart publish command to warn in this case?

--
For other discussions, see https://groups.google.com/a/dartlang.org/

For HOWTO questions, visit http://stackoverflow.com/tags/dart

To file a bug report or feature request, go to http://www.dartbug.com/new

To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

[Günter Zöchbauer]

If you consider the added feature a breaking change and bump the matcher version to 0.12.0 instead of 0.11.2 this would solve the problem, wouldn't it?

[Alexandre Ardhuin]

If I correctly understand this should be only applicable if the export is global (with `export 'xxx';`).

If my lib selects exactly what is exported (with `export 'xxx' show yyy;`) there doesn't seem to be any problem.

Is it right ?

Alexandre

[Lasse R.H. Nielsen]

On Wed, Dec 10, 2014 at 6:59 AM, Günter Zöchbauer <gzo...@gmail.com> wrote:

If you consider the added feature a breaking change and bump the matcher version to 0.12.0 instead of 0.11.2 this would solve the problem, wouldn't it?

If every change is breaking, no change is. :)

You should not consider an added feature a breaking change if the feature is backwards compatible.

Backwards compatible means: This change alone does not break any existing code.

That's what minor version numbers are for.

If you start marking such changes the same way as breaking changes, then you loose precision, and you get more problems with compatibility, not fewer.

Users of your library will still need a way to know whether your change is really breaking, or it's non-breaking but you incremented the version number anyway, so they can say that they are compatible with multiple major versions.

So, as a general solution: No, it doesn't solve the problem.

/L

--

Lasse R.H. Nielsen - l...@google.com  

'Faith without judgement merely degrades the spirit divine'
Google Denmark ApS - Frederiksborggade 20B, 1 sal - 1360 København K - Denmark - CVR nr. 28 86 69 84

[Lasse R.H. Nielsen]

On Wed, Dec 10, 2014 at 11:55 AM, Alexandre Ardhuin <alexandr...@gmail.com> wrote:

If I correctly understand this should be only applicable if the export is global (with `export 'xxx';`).

If my lib selects exactly what is exported (with `export 'xxx' show yyy;`) there doesn't seem to be any problem.

Is it right ?

No, not in general.

You might be lucky that "yyy" hasn't changed between minor versions, so your own exported API doesn't change, but you can't know that without checking.

Any minor version update of the exported API *might* have changed the part you are exporting, thereby changing your public API, and warranting a minor-version update of your package too. The only way to ensure that is to depend on the exported package at the minor version level, not the major version, so your package won't automatically claim to be compatible with a newer version.

As a general rule: Be very careful about exporting any package that you don't control the updates of. It means you have to react every time that package changes in order to keep the semantic versioning contract - a user visible API change requires a minor-version increment. That, or bind yourself tightly to a minor version of the package, and then risk being incompatible with other uses of the package.

/L

[Alexandre Ardhuin]

Oops I forgot to mention that "yyy" was a function in my case.

Thanks for your advices.

Alexandre

--

[Lasse R.H. Nielsen]

On Wed, Dec 10, 2014 at 12:31 PM, Alexandre Ardhuin <alexandr...@gmail.com> wrote:

Oops I forgot to mention that "yyy" was a function in my case.

Argument still holds, though. It's a backwards compatible change to add an extra optional parameter to a function, so that only requires a minor version update. Same with loosening restrictions on the arguments. If your package exports the function that changes, it should get a minor version update as well.

An alternative is to delegate to the function instead of exporting it:

    Foo yyy(Bar a, Baz b) => xxx.yyy(a, b);

That would not change even if xxx.yyy gets more optional parameters or looser types on the arguments.

Your function will still change behavior if xxx.yyy gets more permissive with the arguments in other ways than with types, e.g., no longer throwing on `null`, but at least it's not something that shows up in your API, and it's not something we generally worry about (because it would be crippling to do so - it would require extra checking every time an argument is passed through to a function from another package).

/L

[Cogman]

I don't get why this is an "edge case".  If they depend on a feature in 0.11.2, then their constraints should be

">=0.11.2 <0.12.0"

The min version constraint should be the minimum version that the library or project needs to function correctly.  If it isn't, then the project is setup incorrectly.

You wouldn't expect a constraint of

">=0.9.0 <0.40.0"

to work correctly.

[Natalie Weizenbaum]

On Wed, Dec 10, 2014 at 3:06 AM, 'Lasse R.H. Nielsen' via Dart Misc <mi...@dartlang.org> wrote:

On Wed, Dec 10, 2014 at 6:59 AM, Günter Zöchbauer <gzo...@gmail.com> wrote:

If you consider the added feature a breaking change and bump the matcher version to 0.12.0 instead of 0.11.2 this would solve the problem, wouldn't it?

If every change is breaking, no change is. :)

You should not consider an added feature a breaking change if the feature is backwards compatible.

Backwards compatible means: This change alone does not break any existing code.

That's what minor version numbers are for.

If you start marking such changes the same way as breaking changes, then you loose precision, and you get more problems with compatibility, not fewer.

Users of your library will still need a way to know whether your change is really breaking, or it's non-breaking but you incremented the version number anyway, so they can say that they are compatible with multiple major versions.

So, as a general solution: No, it doesn't solve the problem.

Also, this requires matcher to know that it's being exported by some other library. In general, a library doesn't know if it's being exported or will be exported in the future.