vendoring related question/issue

146 views
Skip to first unread message

Sachin Puranik

unread,
May 9, 2021, 11:03:58 AM5/9/21
to golang-nuts
Hi Gophers,
I noticed the following issue while vendoring the library.

Preface :
Basically, I created the module, for myself. This module contains rendering templates and HTML files, few test files alongside the code. they are all tightly coupled to my module.

Issue:
when I use this module in another project as a vendor module, I noticed that all the code files are copied well, but all my test files, HTML templates, and files are completely ignored. They are not available in the vendor folder.

Kindly let me know if there is an option to achieve this, or am I doing something wrong?

Thanks in advance,
Best regards,
Sachin.

Sean Liao

unread,
May 9, 2021, 12:17:23 PM5/9/21
to golang-nuts
`go mod vendor` only includes what is needed to build your main module.
1.16 has https://pkg.go.dev/embed so you can embed static assets (such as template/html files) into your final binary
(they will also be available in the vendor directory but I don't think this is your final goal)

Sachin Puranik

unread,
May 10, 2021, 1:00:37 AM5/10/21
to Sean Liao, golang-nuts
Hi Sean,
Thanks for the response. Though I am not sure if the solution satisfies my use case. I am doing some experimentation and will get back to you soon with the findings.

Thanks and Regards,
Sachin.

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/FAIN1fLonUc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/66c0b083-fd53-4c96-8541-60a0fc23a581n%40googlegroups.com.

Sachin Puranik

unread,
Jun 17, 2021, 11:25:22 PM6/17/21
to Sean Liao, golang-nuts
Dear Gophers,
I have some more thoughts about the vendoring issue I observed earlier.

Preface :
Basically while vendoring other than go files, nothing is copied in the vendor folder(except the embed package).
It can be some Readme.MD, or some text file, or HTML templates.


Issue Details:
In the last email, thanks to Sean, she suggested that I use the https://pkg.go.dev/embed package.
But I still see some issues.
1. As per package description, this will work compile-time, which means the files will be embedded in the binary and affect the size of the artifact.
2. Secondly I may want to use these files from the folder for different purposes, ex: - I need not embed them, but as a part of my own build step I collect all such dependencies from my module and move them to the common serve folder from where they are served to the web.
3. it could be literally any purpose.

Ultimately I think it's the purview of developers on how to use those files being part of the package, hence they should not be discarded by applying smart compilation.


Now there is another associated problem, Let's say I add that manually in my vendor folder, when I run  'go mod vendor', all these files are forcibly deleted while updating and I am back to square one.

Can you help to resolve this?

Regards,
Sachin

 

jake...@gmail.com

unread,
Jun 18, 2021, 8:46:09 AM6/18/21
to golang-nuts
I don't use vendor, so I'm just thinking here. If I understand correctly, a "//go:embed" statement will cause the the target file to be included in the vendor. If that is the case, then perhaps you could create a vendored_files.go file that embeds all the files you want vendored, then add a build tag to the file so it never actually gets built into your executable?

I have not actually tried this, but it seems like it should work.

Chris Burkert

unread,
Jun 18, 2021, 10:22:18 AM6/18/21
to Sachin Puranik, Sean Liao, golang-nuts
I believe you should distinguish between the build and the run environment. Your build environment is largely defined by the language and its ecosystem, e.g. the vendor folder has a semantic for the Go tools. After the build you deploy the resulting artifacts to your run environment. That may be the go binary and files in a vendor folder. But the semantics of that vendor folder is different.

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CANE8YwiLfiNw3XeEnPb%3DX0tD6%2B6--THe34YaChjdOLV%2BGWa3KQ%40mail.gmail.com.

Sachin Puranik

unread,
Jun 18, 2021, 1:41:18 PM6/18/21
to jake...@gmail.com, golang-nuts
This seems like a good Idea...I will try and let you know soon.

Thanks,
Sachin.

Sachin Puranik

unread,
Jun 18, 2021, 1:48:51 PM6/18/21
to Chris Burkert, Sean Liao, golang-nuts
This is quite an open answer.
The point I am trying to make is, why not all the files from the external library available in the vendor folder. why only selective. (selection is based on the build). These files can be txt , md , go , go test or zzz...doesn't matter.
Ultimately it's the developer's choice to decide what to do with them ...

Regards,
Sachin.
Reply all
Reply to author
Forward
0 new messages