Why is pdfcpu v0.5.0 290MiB on proxy.golang.org ?

[Tamás Gulácsi]

go get github.com/pdfcpu/pdf...@v0.5.0 hung, so I've investigated:

$ go get -x github.com/pdfcpu/pdfcpu/pkg/api@latest        
# get http://proxy.golang.org/github.com/@v/list                  
# get http://proxy.golang.org/github.com/pdfcpu/pdfcpu/@v/list        
# get http://proxy.golang.org/github.com/pdfcpu/pdfcpu/pkg/api/@v/list        
# get http://proxy.golang.org/github.com/pdfcpu/pdfcpu/pkg/@v/list  
# get http://proxy.golang.org/github.com/pdfcpu/@v/list                
# get http://proxy.golang.org/github.com/@v/list: 404 Not Found (0.145s)
# get http://proxy.golang.org/github.com/pdfcpu/@v/list: 404 Not Found (0.145s)                
# get http://proxy.golang.org/github.com/pdfcpu/pdfcpu/pkg/@v/list: 404 Not Found (0.149s)
# get http://proxy.golang.org/github.com/pdfcpu/pdfcpu/pkg/api/@v/list: 404 Not Found (0.149s)
# get http://proxy.golang.org/github.com/pdfcpu/pdfcpu/@v/list: 200 OK (0.150s)

go: downloading github.com/pdfcpu/pdfcpu v0.5.0                          
# get http://proxy.golang.org/github.com/pdfcpu/pdfcpu/@v/v0.5.0.zip                                                                                           

$ curl -X HEAD -L -v http://proxy.golang.org/github.com/pdfcpu/pdfcpu/@v/v0.5.0.zip

< HTTP/2 302 Found

< Location: https://storage.googleapis.com/proxy-golang-org-prod/98db0a8de358d04c-github.com:pdfcpu:pdfcpu-v0.5.0.zip?Expires=1693035533&GoogleAccessId=gcs-url
signer-prod%40golang-modproxy.iam.gserviceaccount.com&Signature=Z6z%2FSzrSw6HYRQRAlZfRTB36whErbhGl4rVFBnnR%2FRG0J14GUYiFXHsk%2FmMPRJAIqcgdQZ0vND4QQ%2FRlJaS6AE4
RQtwhqDx6pCJn6%2FTPbVUaVBPgEdWppd2x5r1%2BR1eOn54VjE%2BNWZ0LKT9IOCwLN9oWjZPQrz1WnPfKn7vZIc3E5MQd%2FxnZ8foQBfNEJ6WgNFcD6QzUlNRSJkZk8EPa8G7hsAEwZKLwI1GgfIWwtWgd2G
We%2FqpUOqxdPhSorKlJqVGovpVY4n9QTHPRXJGqrXKSaCDZohdIK%2B%2FNklGctIXlK57HNMmzAatyETAOx5kCIfeL3PTxCWszixjy1PkZQA%3D%3D                                          

< HTTP/2 200
< x-guploader-uploadid: ADPycdv391ZoUD64eO_-_QY6cAnAFZIdoaseg8u0fxxTTCD9kyNMn8g8cYd_mB3k1HNHMBOF_dxn9d36p_hNjHbTCYaCOw
< date: Fri, 25 Aug 2023 07:47:24 GMT
< cache-control: public,max-age=3600,must-revalidate
< expires: Fri, 25 Aug 2023 08:47:24 GMT
< last-modified: Sun, 20 Aug 2023 12:49:28 GMT
< etag: "2396accaf05435436ab40e7eee3700f1"
< x-goog-generation: 1692535768254742
< x-goog-metageneration: 1
< x-goog-stored-content-encoding: identity
< x-goog-stored-content-length: 293287851
< content-type: application/zip
< content-disposition: attachment; filename="v0.5.0.zip"
< x-goog-hash: crc32c=61bZnw==
< x-goog-hash: md5=I5asyvBUNUNqtA5+7jcA8Q==
< x-goog-storage-class: MULTI_REGIONAL
< accept-ranges: bytes
< content-length: 293287851
< server: UploadServer
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

??

v0.4.2 is just 3MiB.

[Axel Wagner]

ISTM that's because they include a lot of PDFs for samples and test data in their repository now:

https://github.com/pdfcpu/pdfcpu/tree/master/pkg/samples (245 MB)

https://github.com/pdfcpu/pdfcpu/tree/master/pkg/testdata (77 MB)

This isn't due to the module mirror doing anything weird, it's just that the module is now big because it includes tons of data it didn't use to.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/00c8d89f-5d7c-47f7-810b-68ee6046b995n%40googlegroups.com.

[Tamás Gulácsi]

You're right, thanks.

Do you have any simple solution for this (beside deleting those files)?

Putting such files in a git submodule or git-LFS seems appropriate but complex.

[Axel Wagner]

Not sure if it's a good idea, but you could put them in a separate Go module and only import that from tests. That way they *should* not get downloaded when the module is only imported as a dependency. I think. OTOH you'd have to use `go:embed` to export them, which means every `go test` has to link in a couple hundred MB of data, which is also not great. Alternatively, the tests could download and unpack them themselves - though that is less hermetic, there is a danger of the downloaded version of the test data and the one the test expect drifting apart.

Really, the best would probably be to try and reduce the number and/or size of those files.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8b0f6828-ab41-4f82-a117-d6a11f4b827bn%40googlegroups.com.