encoding an integral float like 1.0 as "1.0" in JSON (not "1"), and other JSON questions

[Basile Starynkevitch]

Hello All,

This could be related to issue #6384

I want to JSON encode a float64 point number always as a JSON float, so 1.0 should be encoded as 1.0 not as 1 because at decoding time I want to distinguish them.
Conceptually, I am persisting in JSON format something which is somehow close to some AST, or even to S-expressions (see below for details). So I want a float 1.0 to be different of the integer 1.

If you are curious, my exact code is commit 096ec00f53011b2a1b05ce on github, and when running go test -v objvalmo with my monimelt project being last in my GOPATH I'm getting as output (amongst many other lines)

json_emit v=-1.000000 of type objvalmo.FloatV
json_emit buf: -1

and I would like that second line to be at least json_emit buf: -1.0 because I need to separate objvalmo.FloatV from objvalmo.IntV types

I played with the following json/encoding example and changed it (by adding some mass being 1.0) to

package main

import (
    "encoding/json"
    "fmt"
    "os"
)

func main() {
    h := json.RawMessage(`{"precomputed": true}`)

    c := struct {
        Header *json.RawMessage `json:"header"`
        Body   string           `json:"body"`
        Mass   float64            `json:"mass"`
    }{Header: &h, Body: "Hello Gophers!", Mass: 1.0}

    b, err := json.MarshalIndent(&c, "", "\t")
    if err != nil {
        fmt.Println("error:", err)
    }
    os.Stdout.Write(b)

}

and expected the output to contain "mass" : 1.0 but it still gives a line with only  "mass": 1 without any decimal point. Is there any way to change that and get that decimal point?


More generally, I don't understand well how should I use JSON facilities in Go. IMHO, there are two approaches: one is using go reflection facilities, and claim to be able to serialize most arbitrary values. another is not using them at all (and this is what I would prefer) by constructing some explicit data in memory mimicking JSON format. FWIW, I care a bit about performance.

To explain my issues, let us imagine that I want to serialize (and unserialize) in JSON format some s-expressions (or some program of a micro Scheme or micro Lisp interpreter), with both lists and vectors as composite types, with the additional constraint that some symbols should not be serialized (and we'll output the null JSON value for them instead) For simplicity assume that we have only alphabetical symbols like xy foo bar efg Let us suppose that we want to ignore all symbols starting with a vowel  (so we ignore efg here). so (foo 1 #(2 xy)  (bar "ab" efg -1.0 "cd")) is an example of such s-expr.. It is a list of 4 elements. The third element is a vector #(2 xy) of 2 components. The 4th and last element is a list (bar "ab" efg -1.0 "cd") of 5 elements whose 3rd element is the symbol efg which, since starting with a vowel, should be ignored and serialized as null and whose 4th element is the floating point -1.0 (not the integer -1). I would like to serialize that S-expr value as e.g.

{ "list" : [ { "symb" : "foo" }, 1, { "vect" : [ 2, { "symb" : "xy" } ] }, { "list" : [ { "symb" : "bar" }, null, -1.0, "cd" ] } ] }

Actually, I am not exacty coding a microScheme. I haved shared objects (of *ObjectMo type) and values (of ValueMo type) inside objects (every object has a map and a slice of values). I want to persist them in JSON inside an Sqlite database, but some objects are transient and non-persistent (so I would encode them as null). All objects have a unique id like _0ECuE7b9XhQ_3HvWl3RGhD6

Any advices are welcome. In particular, I am a bit afraid of the reflection approach (because I want serialization & deserialization to not be too slow, and I really need to master the JSON representation).

[Jakob Borg]

On 2 Mar 2017, at 18:29, Basile Starynkevitch <bas...@starynkevitch.net> wrote:

I would like that second line to be at least json_emit buf: -1.0 because I need to separate objvalmo.FloatV from objvalmo.IntV types

JSON itself doesn't distinguish between integers and floats so it could be a bad fit for your use case. That said, since you have custom types you could implement FloatV.MarshalJSON (or MarshalText) and do the formatting there as you please. 

//jb 

[Konstantin Khomoutov]

On Thu, 2 Mar 2017 02:29:22 -0800 (PST)
Basile Starynkevitch <bas...@starynkevitch.net> wrote:

> I want to JSON encode a float64 point number always as a JSON float,
> so 1.0 should be encoded as 1.0 not as 1 because at decoding time I
> want to distinguish them.

[...]

Use custom type with a custom marshaler [1]:

------------>8------------

package main

import (
"encoding/json"
"fmt"

"math"
"os"
)

type myFloat64 float64

func (mf myFloat64) MarshalJSON() ([]byte, error) {
const ε = 1e-12
v := float64(mf)
w, f := math.Modf(v)
if f < ε {
return []byte(fmt.Sprintf(`%v.0`, math.Trunc(w))), nil
}
return json.Marshal(v)
}

type data struct {

Header *json.RawMessage `json:"header"`
Body string `json:"body"`

Mass myFloat64 `json:"mass"`

}

func main() {
h := json.RawMessage(`{"precomputed": true}`)

for _, item := range []data{
{

Header: &h,
Body: "Hello Gophers!",

Mass: 1.0,
},
{
Header: &h,
Body: "Holã Gophers!",
Mass: 1.42,
},
} {
b, err := json.MarshalIndent(&item, "", "\t")

if err != nil {
fmt.Println("error:", err)
}
os.Stdout.Write(b)
}
}

------------>8------------

Which outputs:

{
"header": {
"precomputed": true
},
"body": "Hello Gophers!",
"mass": 1.0
}{
"header": {
"precomputed": true
},
"body": "Holã Gophers!",
"mass": 1.42
}

I'm not truly sure about the usage of epsilon: maybe it's OK for your
case to just compare the fractional part with float64(0).

1. https://play.golang.org/p/d9cTkr70sJ

[Basile Starynkevitch]

On Thursday, March 2, 2017 at 12:01:49 PM UTC+1, Jakob Borg wrote:

On 2 Mar 2017, at 18:29, Basile Starynkevitch <bas...@starynkevitch.net> wrote:

I would like that second line to be at least json_emit buf: -1.0 because I need to separate objvalmo.FloatV from objvalmo.IntV types

JSON itself doesn't distinguish between integers and floats so it could be a bad fit for your use case.

In theory you are right, json.org don't make the difference. In practice, a lot of JSON libraries (e.g. jansson for C, jsoncpp for C++,  YoJson for Ocaml)  are distinguishing them, in particular because they are different types at the hardware level.
Separating floats & integers is so common in most JSON implementation that I believe it is an implicit part of the JSON spec (I know it is not, but JSON parsers are distinguishing numbers with or without decimal points). Actually, I can't name any JSON implementation confusing integers and floats (but of course there might be, it even looks that YoJson might be configured as such, but probably is not by default, because int and float are different basic types in Ocaml).

 

That said, since you have custom types you could implement FloatV.MarshalJSON (or MarshalText) and do the formatting there as you please. 

Thanks for the hint. Will try that (not very simple for me to understand how).

//jb 

[Konstantin Khomoutov]

On Thu, 2 Mar 2017 03:35:39 -0800 (PST)
Basile Starynkevitch <bas...@starynkevitch.net> wrote:

[...]

> > JSON itself doesn't distinguish between integers and floats so it
> > could be a bad fit for your use case.

[...]

> In theory you are right, json.org don't make the difference. In
> practice, a lot of JSON libraries (e.g. jansson

> <https://jansson.readthedocs.io/en/2.9/apiref.html#number> for C,
> jsoncpp
> <http://open-source-parsers.github.io/jsoncpp-docs/doxygen/class_json_1_1_value.html>
> for C++, YoJson <https://github.com/mjambon/yojson> for Ocaml) are

> distinguishing them, in particular because they are different types
> at the hardware level.

I think you're focusing on an irrelevant side of this equation: when
a JSON library does unmarshaling, it either "knows" the schema of the
data it unmarshals -- either explicitly or, say, through reflection
over the type of the receiving variable or through some other means, --
and in this case the library interprets the JSON value it reads in a
way to fullfill the receiver's data type, or the schema is not known --
say, we're dealing with unmarshaling into a generic container type like
map[string]interface{} or with a typeless language, and then the data
type of the resulting value is usually inferred from the data type of
the source JSON value.

Both cases happen in the wild, but to state that only the latter case
is "the correct" approach to parsing JSON is IMO incorrect.

Say, when you would use encoding/json to unmarshal that literal 1 into
a field of type float64, Go would do the right thing and produce a
float64(1) value.

[...]

[Basile Starynkevitch]

Thanks for all for the help. I managed to commit 4da8b9c7a7d7822ca1e45ce660

[Thomas Bushnell, BSG]

There is no such distinction in json as a float vs. an int; they are just numbers. This is essentially a consequence of the javascript underpinnings of json.

Thomas

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.