expectations for generics in Go 1.18
29,036 views
Skip to first unread message
Russ Cox
Oct 27, 2021, 4:56:12 PM10/27/21
to golang-dev
Hi all,
Absent an unexpected serious problem, Go 1.18 will include support for generics. Generics are the most significant change to Go since the release of Go 1, and certainly the largest single language change we've ever made. This mail explains a bit what the inclusion of generics does and does not mean for us and for users.
Any new Go feature, whether in the language or the library, carries uncertainty with it: uncertainty about how to use it, uncertainty about how not to use it, and uncertainty about what subtle bugs have gotten past the existing tests. Generics is not immune to this uncertainty; in fact, because it is a large new feature, the uncertainty is correspondingly larger too.
Because we will not know what the best practices are for using generics, our docs will not be able to give precise, definitive answers about when to use them and when not to. We can and will still give rough guidelines. For comparison, we wrote the initial version of Effective Go after an intense full year of writing Go code non-stop. We don't have the same level of experience yet with generics, so while we will of course provide docs about how to use generics, we can't provide anything as prescriptive about style and best practices. We simply don't know them yet.
Because we will not know what the best practices are for writing generic packages, the initial generic code we release—in particular, the maps and slices packages that went through the proposal process—will land first in golang.org/x/exp, where there is no guarantee of backwards compatibility. Once we have more experience, we hope to promote some of those packages to the standard library. The exception is the constraints package, which is fundamental to writing certain generic code and will be added to the standard library in Go 1.18.
Because we will not have any production experience with generics, we will make clear in the release notes that production uses of generics should be approached with appropriate caution. This is not a criticism of the team’s excellent work. It is simply an observation that generics is different from most Go changes. When we rewrite the garbage collector or change the calling convention, we run all of Google’s Go programs, in test and in production, using the new implementation, which exercises the change quite a bit and shakes out hard-to-find bugs. In contrast, rebuilding non-generic code with the in-progress Go 1.18 toolchain does not exercise the support for generics, meaning we can't build the same confidence.
All that said, Go 1.18 carries with it the same backwards compatibility promise that other Go 1.x releases have: we will not break code that builds with Go 1.18, including code using generics. In the worst case, if we discover some fatal problem with the Go 1.18 semantics and need to change them (in, say, Go 1.19), we will use the go.mod file's go line to determine whether the source files within that module expect Go 1.18 or Go 1.19+ semantics. (We do not anticipate needing to do that!)
We expect that some package authors will be eager to adopt generics. If you are updating your package to use generics, please consider isolating the new generic API into its own file, build-tagged for Go 1.18 (//go:build go1.18), so that Go 1.17 users can keep building and using the non-generic parts.
It is also worth noting that third-party tools may not support generics in full at the Go 1.18 release. We are talking to the many tools authors and trying to make sure they get updated appropriately, but individual tools each have their own timelines.
One common question we've gotten is: given all this uncertainty, why not make generics opt-in? The answer is that, at this point, the only way to reduce the uncertainty is to make generics available by default. When we made vendoring opt-in back in Go 1.5, what happened was that almost no one really used it until Go 1.6 when it was turned on by default, and so the Go 1.5 release did nothing to reduce our uncertainty about how well vendoring would work for people. On the other hand, the Go 1.5 release definitely bifurcated the ecosystem into “code that worked with standard Go” and “code that worked with vendoring enabled.” We want to avoid that outcome here, as much as possible.
The most important thing everyone here can do is write some generic code and let us know if you find bugs, unclear compiler errors, and so on. I have written some generic data structures recently and am very happy with the overall experience. I hope you will be too; please file bugs if not. Thanks!
Best,
Russ
Absent an unexpected serious problem, Go 1.18 will include support for generics. Generics are the most significant change to Go since the release of Go 1, and certainly the largest single language change we've ever made. This mail explains a bit what the inclusion of generics does and does not mean for us and for users.
Any new Go feature, whether in the language or the library, carries uncertainty with it: uncertainty about how to use it, uncertainty about how not to use it, and uncertainty about what subtle bugs have gotten past the existing tests. Generics is not immune to this uncertainty; in fact, because it is a large new feature, the uncertainty is correspondingly larger too.
Because we will not know what the best practices are for using generics, our docs will not be able to give precise, definitive answers about when to use them and when not to. We can and will still give rough guidelines. For comparison, we wrote the initial version of Effective Go after an intense full year of writing Go code non-stop. We don't have the same level of experience yet with generics, so while we will of course provide docs about how to use generics, we can't provide anything as prescriptive about style and best practices. We simply don't know them yet.
Because we will not know what the best practices are for writing generic packages, the initial generic code we release—in particular, the maps and slices packages that went through the proposal process—will land first in golang.org/x/exp, where there is no guarantee of backwards compatibility. Once we have more experience, we hope to promote some of those packages to the standard library. The exception is the constraints package, which is fundamental to writing certain generic code and will be added to the standard library in Go 1.18.
Because we will not have any production experience with generics, we will make clear in the release notes that production uses of generics should be approached with appropriate caution. This is not a criticism of the team’s excellent work. It is simply an observation that generics is different from most Go changes. When we rewrite the garbage collector or change the calling convention, we run all of Google’s Go programs, in test and in production, using the new implementation, which exercises the change quite a bit and shakes out hard-to-find bugs. In contrast, rebuilding non-generic code with the in-progress Go 1.18 toolchain does not exercise the support for generics, meaning we can't build the same confidence.
All that said, Go 1.18 carries with it the same backwards compatibility promise that other Go 1.x releases have: we will not break code that builds with Go 1.18, including code using generics. In the worst case, if we discover some fatal problem with the Go 1.18 semantics and need to change them (in, say, Go 1.19), we will use the go.mod file's go line to determine whether the source files within that module expect Go 1.18 or Go 1.19+ semantics. (We do not anticipate needing to do that!)
We expect that some package authors will be eager to adopt generics. If you are updating your package to use generics, please consider isolating the new generic API into its own file, build-tagged for Go 1.18 (//go:build go1.18), so that Go 1.17 users can keep building and using the non-generic parts.
It is also worth noting that third-party tools may not support generics in full at the Go 1.18 release. We are talking to the many tools authors and trying to make sure they get updated appropriately, but individual tools each have their own timelines.
One common question we've gotten is: given all this uncertainty, why not make generics opt-in? The answer is that, at this point, the only way to reduce the uncertainty is to make generics available by default. When we made vendoring opt-in back in Go 1.5, what happened was that almost no one really used it until Go 1.6 when it was turned on by default, and so the Go 1.5 release did nothing to reduce our uncertainty about how well vendoring would work for people. On the other hand, the Go 1.5 release definitely bifurcated the ecosystem into “code that worked with standard Go” and “code that worked with vendoring enabled.” We want to avoid that outcome here, as much as possible.
The most important thing everyone here can do is write some generic code and let us know if you find bugs, unclear compiler errors, and so on. I have written some generic data structures recently and am very happy with the overall experience. I hope you will be too; please file bugs if not. Thanks!
Best,
Russ
Frederik Zipp
Nov 7, 2021, 3:40:12 AM11/7/21
to golang-dev
the maps and slices packages that went through the proposal process—will land first in golang.org/x/exp, where there is no guarantee of backwards compatibility.
Can we still expect them to be published under x/exp before the final 1.18 release, say along with the beta release?
Reply all
Reply to author
Forward
0 new messages