The color picker seems to have trouble with saving and loading configuration data consistently.
When I pick an 8-bit ANSI compatible color like #afd700, then iTerm2 corrupts this to #aed700.
This happens between closing the color picker and reopening it.
This happens with sRGB. In fact, the color picker frequently forces the color space label back to P3, whereas other applications persist the preferred color space.
This happens with the cursor background swatch, for example.
Latest stable iTerm2 (v3.6.6) via Homebrew. macOS Tahoe.