XML-XMLParser Conflict

[Sean DeNigris]

XML-XPath declares its dependency to XML-XMLParser as follows: github://pharo-contributions/XML-XMLParser:v3.6.x/src. This causes a conflict with Soup, which declares it this way: github://pharo-contributions/XML-XMLParser. This seems to necessitate special handling to load any project that depends on both. 

What's the best solution here? Remove the 3.6.x from one? Add it to the other? Something else?...

FYI I don't want to handle this with a lock or conflict block because then every project which depends on these often-used-together projects will need one. I'd like this combination to just load cleanly.

[Dale Henrichs]

Sean,

IIRC, XML-XPath seems to want a particular version of XML-XML-Parser ... so making changes (if any) that are needed for XML-XPath to run with the "latest version of XML-XMLParser is probably the better path to follow ... knowing why v3.6.x was required is a good question to ask whoever made that change in the first place ... were they avoiding bug in later versions? or was it simply "too hard at the moment" to port to a later version ? or ???

Dale

[Sean DeNigris]

Apparently, there is a line of thought in the Pharo community that dependencies should be partially pinned (excluding patch versions) by default, as described in the pull request and spilling over to GT Discord, starting with this message. This doesn't seem right to me for several reasons unless we're talking about a tagged release that should be reproducible because:

• We have limited community resources to manage these pinnings
• pinnings seem somewhat arbitrary because often community maintainers are not intimately familiar with the project
• seems easier to just react to CI failures; defensive pinning smells like premature optimization IMO
• They are pinning the minor version, so builds will not be reproducibly anyway because the patch is floating
• what does reproducibility even mean from an untagged baseline?

The last major problem is that if we were going to pin for compatibility guarantees based on semantic versioning, it would seem we should pin the major version, not the minor, but trying this (e.g. `github://pharo-contributions/XML-XMLParser:v3.x` leads to `NotFound: revspec 'v3.x' not found` (see failed build here)

[Sean DeNigris]

After `v3.x.x` didn't work either, I read on the Pharo wiki that "{version}… can be… a tag like… 'v1.x.x'". I then noticed that XMLParser defines tags like v3.5.x and v3.6.x.

This blew my mind because I had thought x was like a regex wildcard, but now it seems like one can't take advantage of loose semantic versions (e.g. pinning major only) unless the dependency itself has defined these tags. Is that right?

So that I can fix this specific case I'm faced with (but not generally encouraging as a solution), if I were to create tags to pin only the major version, would v3.x.x or v3.x be better?

Thanks,

Sean

[Gabriel Cotelli]

AFAIR there's no magic. You need a tag or branch for the major versions if you want to use this kind of scheme. 

In the ba-st organization, we use the following conventions:

- Tags are used for releases with a triplet of vMAJOR.MINOR.PATCH increasing the major when some breaking changes are made

- We keep a branch for every major version that moves from one commit to another when a new release of this major is made, so for example, if we have the v3.1.0 and v3.2.0 tags, there's also a v3 branch pointing to the same commit that v3.2.0. If we later produce a v3.2.1 the v3 is moved to this same commit. So, tags are immutable, branches are moving.

- Dependencies between projects are specified using the major branches. Initially, we tried to use specific versions (the tags) as dependencies but the cascade of changes you have to make when changing something was just not worth the effort and we lacked the manpower to maintain this scheme. So, using the branches in the dependencies is the compromise we got, losing full build reproducibility as a side effect. For people wanting a fully reproducible build, they can lock pinned versions with Metacello before the loading for each of the dependencies.

[Sean DeNigris]

I prefer not to cross-post, but since there didn't seem to be resolution here, I posted the following to the Pharo-Dev ML:

I recently experienced some CI problems for several of my projects because of an (IMO unnecessarily tight) version specification of XML-XPATH’s dependency on XMLParser: github://pharo-contributions/XML-XMLParser:v3.6.x/src. At minimum, it seems the dependency should be on the major version (e.g. v3), not the minor (unless I’m missing a needed feature that was added in 3.6).

1. More importantly though, it raises questions about what the best practice is here. In summary, it seems to me that in general it’s better to only pin versions in reaction to CI failures because pinning has significant downsides and uncertainties. Defensive pinning smells like premature optimization IMO. In more detail, partially pinning (to major or minor version) by default doesn't seem right to me for several reasons unless we're talking about a tagged release that should be reproducible because: We have limited community resources to manage these pinnings, which create cascading conflict problems with all other dependent projects. Pinnings seem somewhat arbitrary because often community maintainers are not familiar enough with both projects or have enough time to do a thorough investigation. How does the maintainer know whether a particular dependency version really “works” with the project? Unless they are intimately familiar with both projects (and even then I wouldn’t have confidence in a manual review), the best way I can think of is to rely on passing CI, and in that case… It seems easier to just react to CI failures; defensive pinning smells like premature optimization IMO Partial pinning (minor or major version) will not lead to reproducible builds because the patch is floating What does reproducibility even mean from an untagged baseline? Looking through a bunch of repos, it doesn’t seem that there is consensus either way. Some specify baselines and some specific versions.
2. p.s. for tracking the major, I like ba-st’s naming convention of v{integer} instead of adding “.x”’s of unclear value