Version metadata on Racket packages
Jeff Henrikson
Hello racket-users,
As far as I can tell from reading the Racket documentation, the
racket package manager has only one field to indicate version. I
am trying to understand the intended use of the version field.
Here the documentation seems to recommend that packages use the
version field to denote a semantic versioning scheme of for the
specific package's release cycle:
https://docs.racket-lang.org/pkg/Package_Concepts.html
a version — a string of the form ‹maj›.‹min›, ‹maj›.‹min›.‹sub›, or ‹maj›.‹min›.‹sub›.‹rel›, where ‹maj›, ‹min›, ‹sub›, and ‹rel› are all canonical decimal representations of natural numbers, ‹rel› is not 0, ‹sub› is not 0 unless ‹rel› is supplied, ‹min› has no more than two digits, and ‹sub› and ‹rel› have no more than three digits. A version is intended to reflect available features of a package, and should not be confused with different releases of a package as indicated by the checksum.
Here the documentation refers to a "Racket version number," which
might be taken to mean a version number of the Racket language:
https://docs.racket-lang.org/pkg/catalog-protocol.html
8.1 Remote and Directory Catalogs
In the case of a remote URL or a local directory naming a package catalog, the URL/path is extended as follows to obtain information about packages:
pkg and ‹package› path elements, where ‹package› is a package name, plus a version=‹version› query (where ‹version› is a Racket version number) in the case of a remote URL.
It seems natural for packages that tightly depend on quickly
evolving features of the Racket language to have a versioning
scheme coupled to Racket's own versions. On the other hand, for
packages that work with a variety of possible Racket versions, it
seems to make sense that said packages would have their own
release cycles, compatibility aspirations, and semantic versioning
contract.
Note that some package managers for evolving languages have two
versioning coordinates, one for the language version and one for
the package version.
Appended below this message is a small racket program that
inspects data from the https://pkgs.racket-lang.org/pkgs-all HTTP
endpoint. According to this analysis, 21% of published Racket
packages version using version numbers of the Racket language.
Fewer than 1% of published Racket packages seem to version using
their own version scheme. And 78% of published Racket packages
have only ever filled the version field with "default".
As the documentation notes, using checksum instead of a version
field is sort of like saying any feature can change at any time to
any degree: "A version is intended to reflect available features
of a package, and should not be confused with different releases
of a package as indicated by the checksum."
My question is this: What is the intended use of the version field
in the Racket package manager?
Thanks in advance,
Jeff Henrikson
#lang racket
;; obtain data with:
;; curl https://pkgs.racket-lang.org/pkgs-all > contrib_pkgs-all.sexp
;; relfin is "contrib_pkgs-all.sexp" or similar.
;;
;; In practice it's one value that comes over the wire, so
;; we'll just take the car now.
(define (read-pkgs5 relfin)
(letrec (
(fin (open-input-file relfin))
(iter (lambda (xs)
(let ((x (read fin)))
(if (not (equal? x eof))
(iter (cons x xs))
(begin
(close-input-port fin)
xs))))))
(car (reverse (iter '())))))
;;; Get the versions of a package hash
(define (versions-of-pkg pkg)
(hash-keys (hash-ref pkg 'versions)))
;;; Is there at least one version that appears to be distinct
;;; from a racket version?
;; Racket version numbers contemporaneous with the remote catalog
;; seem to start around version 6.
(define (package-uses-version-for-package? pkg)
(not (empty? (filter (lambda (v)
(and
(string? v)
(string<? v "5")))
(versions-of-pkg pkg)))))
;;; Is there at least one version that appears to be a racket version?
;; Racket version numbers contemporaneous with the remote catalog
;; seem to start around version 6.
(define (package-uses-version-for-racket? pkg)
(not (empty? (filter (lambda (v)
(and
(string? v)
(not (equal? v "default"))
(string<? "5" v)))
(versions-of-pkg pkg)))))
;;; Histogram the identified uses of the version field
(define (classify-version-use pkgs)
(define (how-used pkg)
(cond
((package-uses-version-for-package? pkg) 'package-versioned)
((package-uses-version-for-racket? pkg) 'racket-versioned)
(else 'no-versioning-used)))
(define (increment x)
(+ x 1))
(let* (
(h (make-hash))
(_ (for-each (lambda (kv)
(hash-update! h (how-used (cdr kv)) increment 0))
(hash->list pkgs)))
(num-by-use (hash->list h)))
num-by-use))
(module+ main
(define pkgs (read-pkgs5 "contrib_pkgs-all.sexp"))
(classify-version-use pkgs)
;; '((no-versioning-used . 1472) (racket-versioned . 379) (package-versioned . 13))
)
Sage Gerard
The last sentence of the first paragraph you quoted starts with "A version is intended," but I don't know for sure if it is a complete answer. You'll also notice on the catalog that there are version exceptions, but you may have already read that by now.
Beyond what the manual says, the version field really doesn't mean much in the sense I think you're hinting at. There is no query language where you can ask for, say, the latest version within those available within 4.2.x, for example. Even if there was, it wouldn't be that useful since the default catalog does not maintain artifacts (Bogdan's racksnaps improves on this situation).
Also remember that it is the collection, not the package, is the real "unit of exchange" in the Racket ecosystem. If you release a version 2.0 that includes a breaking change in a `foo` collection, it will conflict with any version of any package published by anyone that happens to define modules of the same name in the same collection. There are reasons for this, but that's out of the scope for your question.--
You received this message because you are subscribed to the Google Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/cde363b4-f0c4-eb6f-c68f-7033bfa14c3c%40gmail.com.
-- ~slg
Philip McGrath
'versions (optional) — a hash table mapping version strings and 'default to hash tables, where each version-specific hash table provides mappings to override the ones in the main hash table, and 'default applies to any version not otherwise mapped.
Clients of a remote catalog may request information for a specific version, but they should also check for a 'versions entry in a catalog response, in case a catalog with version-specific mappings is implemented as a directory or by a file-serving HTTP server. A 'default mapping, meanwhile, allows the main hash table to provide information that is suitable for clients at version 5.3.6 and earlier (which do not check for 'versions).
To make supporting multiple versions of Racket easier, the package catalog software supports version exceptions. Version exceptions allow package authors to specify alternative package sources to be used when installing a given package using a specific version of Racket.
For example, a package that uses on Racket 6.0-specific features could provide a version exception for Racket 5.3.6 using a different branch or tag in the package’s GitHub repository, or a different zip archive, as package source. Users installing the package from Racket 6.0 will use the default source for the package, while those using Racket 5.3.5 will install from the alternative branch, tag, or archive.
A version is intended to reflect available features of a package, and should not be confused with different releases of a package as indicated by the checksum.
(define version "0.1")
(define deps'("base"["B" #:version "0.1"]))
It seems natural for packages that tightly depend on quickly evolving features of the Racket language to have a versioning scheme coupled to Racket's own versions. On the other hand, for packages that work with a variety of possible Racket versions, it seems to make sense that said packages would have their own release cycles, compatibility aspirations, and semantic versioning contract.
Jeff Henrikson
Thanks for the help.
It seems that the issue you raise applies more generally than to /pkgs-all. See appended below curl commands that show that the package version is also missing in the response to a request for information on a specific package.In fact there are two concepts here, and part of what I think confused you is something I just discovered this week and plan to report as a bug: https://pkgs.racket-lang.org/pkgs-all does not include the package's version . . .
Could it be that the package service is actually deleting old versions of packages when new ones are uploaded?
Jeff
curl https://raw.githubusercontent.com/florence/cover/release/cover-lib/info.rkt
#lang info
(define collection 'multi)
(define version "3.3.3")
(define pkg-desc "A code coverage library -- implementation")
(define deps '("base"
"compiler-lib"
"custom-load"
"data-lib"
"errortrace-lib"
"syntax-color-lib"
"testing-util-lib"))
curl https://pkgs.racket-lang.org/pkg/cover-lib#hasheq((author . "spe...@florence.io") (authors . ("spe...@florence.io")) (build . #hash((conflicts-log . #f) (dep-failure-log . #f) (docs . ()) (failure-log . #f) (min-failure-log . #f) (success-log . "server/built/install/cover-lib.txt") (test-failure-log . #f) (test-success-log . "server/built/test-success/cover-lib.txt"))) (checksum . "ad50ffa8f6246053bec24b39b9cae7fad1534373") (checksum-error . #f) (collection . (multi)) (conflicts . ()) (date-added . 1582684086) (dependencies . ("base" "compiler-lib" "custom-load" "data-lib" "errortrace-lib" "syntax-color-lib" "testing-util-lib")) (description . "A code coverage tool, implementation part") (implies . ()) (last-checked . 1615994547) (last-edit . 1582684243) (last-updated . 1582839053) (modules . ((lib "cover/strace.rkt") (lib "cover/cover.rkt") (lib "cover/private/raw.rkt") (lib "cover/private/format-utils.rkt") (lib "cover/private/file-utils.rkt") (lib "cover/format.rkt") (lib "cover/private/contracts.rkt") (lib "cover/main.rkt") (lib "cover/raco.rkt") (lib "cover/private/shared.rkt") (lib "cover/private/html/html.rkt"))) (name . "cover-lib") (ring . 1) (search-terms . #hasheq((:build-success: . #t) (author:spe...@florence.io . #t) (ring:1 . #t) (testing . #t) (tools . #t))) (source . "https://github.com/florence/cover.git?path=cover-lib#release") (tags . ("testing" "tools")) (versions . #hash((default . #hasheq((checksum . "ad50ffa8f6246053bec24b39b9cae7fad1534373") (source . "https://github.com/florence/cover.git?path=cover-lib#release") (source_url . "https://github.com/florence/cover.git?path=cover-lib#release"))))))
Hi Jeff,
In fact there are two concepts here, and part of what I think confused you is something I just discovered this week and plan to report as a bug: https://pkgs.racket-lang.org/pkgs-all does not include the package's version: that's what's discussed on the Package Concepts page you linked to. The ?version= query parameter corresponds to the `'versions` (plural—in hindsight this should have had a different name!) entry in the hash table, which is documented slightly below the passage you quoted from the section on Remote and Directory Catalogs:
'versions (optional) — a hash table mapping version strings and 'default to hash tables, where each version-specific hash table provides mappings to override the ones in the main hash table, and 'default applies to any version not otherwise mapped.
Clients of a remote catalog may request information for a specific version, but they should also check for a 'versions entry in a catalog response, in case a catalog with version-specific mappings is implemented as a directory or by a file-serving HTTP server. A 'default mapping, meanwhile, allows the main hash table to provide information that is suitable for clients at version 5.3.6 and earlier (which do not check for 'versions).
This field is not a property of the package: it is part of the package catalog. For example, when registering a package at https://pkgs.racket-lang.org, you can optionally enter this data in a field on the web form. The purpose is to implement "version exceptions", which are documented further down the page—but clearly this section should be much more pervasively linked to, because this is quite confusing:To make supporting multiple versions of Racket easier, the package catalog software supports version exceptions. Version exceptions allow package authors to specify alternative package sources to be used when installing a given package using a specific version of Racket.
For example, a package that uses on Racket 6.0-specific features could provide a version exception for Racket 5.3.6 using a different branch or tag in the package’s GitHub repository, or a different zip archive, as package source. Users installing the package from Racket 6.0 will use the default source for the package, while those using Racket 5.3.5 will install from the alternative branch, tag, or archive.
This is very rarely useful, in my experience: I thought I might have used it once, but it now seems I didn't.
With that confusing detour out of the way, on to your actual question: "What is the intended use of the version field in the Racket package manager?"
The summary you quoted is right, but there are some important details:
A version is intended to reflect available features of a package, and should not be confused with different releases of a package as indicated by the checksum.
Let's imagine package A depends on package B. Both are at version 0.0. Now B exports a new function, which A would like to use. If B is well-behaved, it has changed its package info.rkt file to include:(define version "0.1")
Now A can write:(define deps'("base"
["B" #:version "0.1"]))
and `raco pkg` will prompt anyone who installs or updates A but has an old version of B to update B, too.
Note that there are some important differences from the semver systems used by some other package managers: in particular, by design, the only possible version constraint is "at least". When you write:
It seems natural for packages that tightly depend on quickly evolving features of the Racket language to have a versioning scheme coupled to Racket's own versions. On the other hand, for packages that work with a variety of possible Racket versions, it seems to make sense that said packages would have their own release cycles, compatibility aspirations, and semantic versioning contract.
Racket has some strong views about compatibility, both as a language/distribution and in the design of its package manager. Racket has, from my perspective, a fairly remarkable level of commitment to not breaking existing code. The package system is modeled on an os-level package manager, a deliberate choice to move away from the intricate versioning mechanism of the older PLaneT package system. It is focused on getting your files installed in the right place. The expectation is that releases of a package will only add functionality, not remove or break it. A package therefore need only increment the version number if it has added something that someone else wants to ensure is available.
If you make a breaking change, the idea is that you should create a new collection (which may or may not be part of a new package) with a new name: consider scribble/lp2 and scribble/lp.
One of the strongest arguments I've seen for this approach is from a non-Racket context: https://ometer.com/parallel.html (h/t Andy Wingo)
If you are familiar with some other package managers, this can sound very scary, but it has never been a problem at all for me in practice. There is also an escape hatch, because this is a matter of social norms, not something automatically enforced: if you call your package, or part of it, "unstable" or "experimental", like unstable/gui/redex or my adjutor/unstable, or you put a big scary warning in the docs like this (again, one of mine) or this, you can follow whatever kind of support practice you want.
-Philip
On Tue, Mar 16, 2021 at 6:51 PM Jeff Henrikson <jehen...@gmail.com> wrote:
Alex Harsányi
Jeff Henrikson
Thanks for the help.
>> Could it be that the package service is actually deleting old versions of packages when new ones are uploaded?
> metadata about them and pointers to where they are stored.
This is an interesting answer, but it's an answer to a question slightly
different from the one I intended. Please allow me to refine my question:
For a given package, does the package service hold information about
historical versions, or only information about the latest version?
Jeff
> On 3/17/21 11:02 AM, Sam Tobin-Hochstadt wrote:
>> Thanks for the help.
>>
>>> In fact there are two concepts here, and part of what I think confused you is something I just discovered this week and plan to report as a bug: https://pkgs.racket-lang.org/pkgs-all does not include the package's version . . .
>> It seems that the issue you raise applies more generally than to /pkgs-all. See appended below curl commands that show that the package version is also missing in the response to a request for information on a specific package.
>>
>> Could it be that the package service is actually deleting old versions of packages when new ones are uploaded?
> metadata about them and pointers to where they are stored.
>
> Sam
Jeff Henrikson
That answers my question. Thank you.
Jeff
No, pkgs.racket-lang.org doesn't keep track of old checksums (or other old metadata) for packages.
If you want that information (or in fact the actual copies of old packages) then I recommend Bogdan Popa's racksnaps.
Sam
--
You received this message because you are subscribed to the Google Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/a1dc06e9-5b50-36e1-86cb-13bb3a8479e4%40gmail.com.