[website] WIP: _content/doc/jsonv2-migration: encoding/json/v2 migration guide
Michael Pratt (Gerrit)
Michael Pratt voted Commit-Queue+1![]( "Open in Gerrit")
| Commit-Queue | +1 |
Related details
- Chris Hines
- Damien Neil
- Joseph Tsai
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Chris Hines (Gerrit)
Chris Hines added 1 comment![]( "Open in Gerrit")
It's a good start.
I refrained from commenting on a few typos I saw until the overall structure is more settled.
I'd like to see more discussion of migration strategies more in the middle of the two extremes of one option at a time and full switch.
I doubt many projects have comprehensive enough tests to find all the differences. Can we suggest any techniques to enhance tests or use fuzzing or jsonsplit to proactively search for issues?
What about migrating Unmarshal code? That probably deserves some discussion as well.
Related details
- Damien Neil
- Joseph Tsai
- Michael Pratt
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Michael Pratt (Gerrit)
Michael Pratt uploaded new patchset![]( "Open in Gerrit")
- TryBots-Pass: LUCI-TryBot-Result-1 by golang...@luci-project-accounts.iam.gserviceaccount.com
Related details
- Damien Neil
- Joseph Tsai
- Michael Pratt
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Michael Pratt (Gerrit)
Michael Pratt voted and added 1 comment![]( "Open in Gerrit")
Votes added by Michael Pratt
| Commit-Queue | +1 |
It's a good start.
I refrained from commenting on a few typos I saw until the overall structure is more settled.
I'd like to see more discussion of migration strategies more in the middle of the two extremes of one option at a time and full switch.
I doubt many projects have comprehensive enough tests to find all the differences. Can we suggest any techniques to enhance tests or use fuzzing or jsonsplit to proactively search for issues?
What about migrating Unmarshal code? That probably deserves some discussion as well.
It's a good start.
I refrained from commenting on a few typos I saw until the overall structure is more settled.
I'd like to see more discussion of migration strategies more in the middle of the two extremes of one option at a time and full switch.
Thanks for the feedback, I also found the style a bit too informal and meandering.
I've reworked things to be a bit more up front about different approach "all-at-once", "option-by-option" (the two extremes), and jsonsplit.
I doubt many projects have comprehensive enough tests to find all the differences. Can we suggest any techniques to enhance tests or use fuzzing or jsonsplit to proactively search for issues?What about migrating Unmarshal code? That probably deserves some discussion as well.
I've added a note that the same process applies to both Marshal and Unmarshal, but perhaps it is too subtle.
Related details
- Chris Hines
- Damien Neil
- Joseph Tsai
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Jonathan Amsterdam (Gerrit)
Jonathan Amsterdam added 15 comments![]( "Open in Gerrit")
First things first: you don't have to!Start stronger. After "to!": "The encoding/json package will never go away."
Then you can drop the "deprecation...deadline" part, but keep the "indefinitely" for emphasis.
For example, this means that if a type implements custom marshaling behavior with [`encoding/json/v2.MarshalerTo`](/pkg/encoding/json/v2#MarshalerTo), the callers marshaling this type through [`encoding/json.Marshal`](/pkg/encoding/json#Marshal) will still go through the custom marshaler.delete
Though you are not required to migrate, there are several motivations for a migration:several good reasons to do so
First, the new API contains a variety of changes to make it easier to work with.is easier to use
These changes are designed to make `encoding/json/v2` more interoperable with the wider JSON ecosystem and less surprising or error-prone to use.and
That said, these behavior changes are not backwards compatible; some applications may depend on the v1 behavior.no new paragraph
"But they"
If we trivially this program to `encoding/json/v2` by changing the import, the program still compiles.modify?
If this were a new program, using an empty array is likely a nice improvement, but in an existing application downstream consumers of this output may be depending on the presence of `null`., so this change may break them
Depending on the type of application and it's risk tolerance, there are several different ways to approach a v2 migration:its
The the source of the problem is not clear, you may want to use one of the approaches below to help troubleshoot.If
This provides a building block to perform a slower migration rather than changing all behavior at once.path? More like a sequence than collection of blocks??
Alternatively, when troubleshooting these options provide a way to bisect down to the exact breaking behavior change.You're abstracting a fairly complex design choice into a quick sentence. How on earth do I make my program bisect-able like that? Maybe an idea or two here would help.
Having read the jsonsplit section, I think this advice is unnecessary. Just use jsonsplit!
If the output differs, it will report a difference, but still return the v1 value to the caller.It will report any differences, but..
This provides reporting of differences from a production service without impacting user-visible behavior.This allows a production service to report differences without changing its behavior.
We can do a smooth production migration something like:We can migrate our production service smoothly using a procedure like the following:
Related details
- Chris Hines
- Damien Neil
- Joseph Tsai
- Michael Pratt
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Michael Pratt (Gerrit)
Michael Pratt uploaded new patchset![]( "Open in Gerrit")
- TryBots-Pass: LUCI-TryBot-Result+1 by golang...@luci-project-accounts.iam.gserviceaccount.com
Related details
- Chris Hines
- Damien Neil
- Joseph Tsai
- Michael Pratt
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Michael Pratt (Gerrit)
Michael Pratt added 15 comments![]( "Open in Gerrit")
Start stronger. After "to!": "The encoding/json package will never go away."
Then you can drop the "deprecation...deadline" part, but keep the "indefinitely" for emphasis.
Done
For example, this means that if a type implements custom marshaling behavior with [`encoding/json/v2.MarshalerTo`](/pkg/encoding/json/v2#MarshalerTo), the callers marshaling this type through [`encoding/json.Marshal`](/pkg/encoding/json#Marshal) will still go through the custom marshaler.Michael Prattdelete
Done
Though you are not required to migrate, there are several motivations for a migration:several good reasons to do so
Done
First, the new API contains a variety of changes to make it easier to work with.Michael Prattis easier to use
Done
These changes are designed to make `encoding/json/v2` more interoperable with the wider JSON ecosystem and less surprising or error-prone to use.Michael Prattand
Done
That said, these behavior changes are not backwards compatible; some applications may depend on the v1 behavior.no new paragraph
"But they"
Done
If we trivially this program to `encoding/json/v2` by changing the import, the program still compiles.Michael Prattmodify?
Done
If this were a new program, using an empty array is likely a nice improvement, but in an existing application downstream consumers of this output may be depending on the presence of `null`., so this change may break them
Done
Depending on the type of application and it's risk tolerance, there are several different ways to approach a v2 migration:Michael Prattits
Done
The the source of the problem is not clear, you may want to use one of the approaches below to help troubleshoot.Michael PrattIf
Done
This provides a building block to perform a slower migration rather than changing all behavior at once.path? More like a sequence than collection of blocks??
Done
Alternatively, when troubleshooting these options provide a way to bisect down to the exact breaking behavior change.You're abstracting a fairly complex design choice into a quick sentence. How on earth do I make my program bisect-able like that? Maybe an idea or two here would help.
Having read the jsonsplit section, I think this advice is unnecessary. Just use jsonsplit!
I've added a mention of jsonsplit here.
If the output differs, it will report a difference, but still return the v1 value to the caller.It will report any differences, but..
Done
This provides reporting of differences from a production service without impacting user-visible behavior.This allows a production service to report differences without changing its behavior.
Done
We can do a smooth production migration something like:We can migrate our production service smoothly using a procedure like the following:
- Chris Hines
- Damien Neil
- Jonathan Amsterdam
- Joseph Tsai
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Jonathan Amsterdam (Gerrit)
Jonathan Amsterdam voted Code-Review+2![]( "Open in Gerrit")
- Chris Hines
- Damien Neil
- Joseph Tsai
- Michael Pratt
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Chris Hines (Gerrit)
Chris Hines added 4 comments![]( "Open in Gerrit")
I like the changes since my last review. I saw a few things while reading the latest version.
This guide describes why you may want to migration from the v1 to v2 package, and the mechanics of doing so safely.migrate
By setting [`AutoDetectOptions`](https://pkg.go.dev/github.com/go-json-experiment/jsonsplit#Codec), `jsonsplit` will even automatically determine which specific options cause the difference.Consider mentioning that AutoDetectOptions is slow (since we are talking about using it in production services in this section) and that jsonsplit provides ways to only perform the extra work on some marshal calls.
1. Switch callsites to `jsonsplit`, set [`CallBothButReturnV1`](https://pkg.go.dev/github.com/go-json-experiment/jsonsplit#CallMode), [`AutoDetectOptions`](https://pkg.go.dev/github.com/go-json-experiment/jsonsplit#CallMode) (optional), and wire up your preferred monitoring approach to [`ReportDifference`](https://pkg.go.dev/github.com/go-json-experiment/jsonsplit#CallMode) (such as logging or published metrics).
2. Monitor your production environment for reported differences.
3. Encode differences.
Where `jsonsplit` reports differences, adjust the options or types to ensure identical output.
For example, in the example above, pass the `json.FormatNilSliceAsNull(true)` option.
In other cases, v2 may report a problem that is straightforward to fix.
For example, applying the "string" JSON struct field tag to an invalid type (such as a struct) is ignored in v1, but reports an error in v2.
While [`ReportErrorsWithLegacySemantics`](/pkg/encoding/json#ReportErrorsWithLegacySemantics) would suppress the error, it makes more sense to drop the "string" tag.
It isn't doing anything anyway.
Note that a difference in output does not necessarily mean that downstream behavior is broken, but that there is an opportunity for breakage.
We adjust options now so we can complete the vast majority of the migration without stopping to evaluate subtle output changes, but after switching to v2, you should revisit these locations to determine if you can migrate to the new behavior.
4. Switch to v2.
Once your production environment stops reporting new differences, you can migrate to v2 behavior by setting [`OnlyCallV2`](https://pkg.go.dev/github.com/go-json-experiment/jsonsplit#CallMode) or [`CallBothButReturnV2`](https://pkg.go.dev/github.com/go-json-experiment/jsonsplit#CallMode) to keep checking for differences.
5. Clean up.This numbered sequence of steps is nice, but it echos the opening paragraphs of this section too much. The beginning of the jsonsplit section has paragraphs starting with "First ...", and "Next ..." which have a sequence of steps narrative structure. Consider rewording the beginning of the section to be less list like and simply describe jsonsplit's capabilities. That leaves the job of describing the migration sequence to this numbered list without it feeling like deja vu when reading it.
Related details
- Damien Neil
- Joseph Tsai
- Michael Pratt
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Michael Pratt (Gerrit)
Michael Pratt uploaded new patchset![]( "Open in Gerrit")
Related details
- Damien Neil
- Joseph Tsai
- Michael Pratt
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Michael Pratt (Gerrit)
Michael Pratt added 3 comments![]( "Open in Gerrit")
This guide describes why you may want to migration from the v1 to v2 package, and the mechanics of doing so safely.Michael Prattmigrate
Done
By setting [`AutoDetectOptions`](https://pkg.go.dev/github.com/go-json-experiment/jsonsplit#Codec), `jsonsplit` will even automatically determine which specific options cause the difference.Consider mentioning that AutoDetectOptions is slow (since we are talking about using it in production services in this section) and that jsonsplit provides ways to only perform the extra work on some marshal calls.
Done
1. Switch callsites to `jsonsplit`, set [`CallBothButReturnV1`](https://pkg.go.dev/github.com/go-json-experiment/jsonsplit#CallMode), [`AutoDetectOptions`](https://pkg.go.dev/github.com/go-json-experiment/jsonsplit#CallMode) (optional), and wire up your preferred monitoring approach to [`ReportDifference`](https://pkg.go.dev/github.com/go-json-experiment/jsonsplit#CallMode) (such as logging or published metrics).
2. Monitor your production environment for reported differences.
3. Encode differences.
Where `jsonsplit` reports differences, adjust the options or types to ensure identical output.
For example, in the example above, pass the `json.FormatNilSliceAsNull(true)` option.
In other cases, v2 may report a problem that is straightforward to fix.
For example, applying the "string" JSON struct field tag to an invalid type (such as a struct) is ignored in v1, but reports an error in v2.
While [`ReportErrorsWithLegacySemantics`](/pkg/encoding/json#ReportErrorsWithLegacySemantics) would suppress the error, it makes more sense to drop the "string" tag.
It isn't doing anything anyway.
Note that a difference in output does not necessarily mean that downstream behavior is broken, but that there is an opportunity for breakage.
We adjust options now so we can complete the vast majority of the migration without stopping to evaluate subtle output changes, but after switching to v2, you should revisit these locations to determine if you can migrate to the new behavior.
4. Switch to v2.
Once your production environment stops reporting new differences, you can migrate to v2 behavior by setting [`OnlyCallV2`](https://pkg.go.dev/github.com/go-json-experiment/jsonsplit#CallMode) or [`CallBothButReturnV2`](https://pkg.go.dev/github.com/go-json-experiment/jsonsplit#CallMode) to keep checking for differences.
5. Clean up.This numbered sequence of steps is nice, but it echos the opening paragraphs of this section too much. The beginning of the jsonsplit section has paragraphs starting with "First ...", and "Next ..." which have a sequence of steps narrative structure. Consider rewording the beginning of the section to be less list like and simply describe jsonsplit's capabilities. That leaves the job of describing the migration sequence to this numbered list without it feeling like deja vu when reading it.
- Chris Hines
- Damien Neil
- Joseph Tsai
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Chris Hines (Gerrit)
Chris Hines voted Code-Review+1![]( "Open in Gerrit")
- Damien Neil
- Joseph Tsai
- Michael Pratt
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Michael Pratt (Gerrit)
Michael Pratt uploaded new patchset![]( "Open in Gerrit")
Related details
- Damien Neil
- Joseph Tsai
- Michael Pratt
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Damien Neil (Gerrit)
Damien Neil voted Code-Review+2![]( "Open in Gerrit")
- Joseph Tsai
- Michael Pratt
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |