unexpected end of JSON input while unmarshal to struct

[diogo.en...@gmail.com]

Could you help me to understand why I'm always getting the error unexpected end of JSON input while trying to unmarshal the following json to the LimitOrder struct? It works if I use golang playground https://play.golang.org/p/udPQ_TayXG but not locally running tests.

P.S.: if I use map[string]json.RawMessage instead of LimitOrder struct I'm able to execute the unmarshal.

{
  "response_data": {
    "order": {
      "order_id": 3,
      "coin_pair": "BRLBTC",
      "order_type": 1,
      "status": 4,
      "has_fills": true,
      "quantity": "1.00000000",
      "limit_price": "900.00000",
      "executed_quantity": "1.00000000",
      "executed_price_avg": "900.00000",
      "fee": "0.00300000",
      "created_timestamp": "1453835329",
      "updated_timestamp": "1453835329",
      "operations": [
        {
          "operation_id": 1,
          "quantity": "1.00000000",
          "price": "900.00000",
          "fee_rate": "0.30",
          "executed_timestamp": "1453835329"
        }
      ]
    }
  },
  "status_code": 100,
  "server_unix_timestamp": "1453835329"
}

LimitOrder struct

type LimitOrder struct {
  OrderId int `json:"order_id"`
  CoinPair string `json:"coin_pair"`
  OrderType int `json:"order_type"`
  Status int `json:"status"`
  HasFills bool `json:"has_fills"`
  Quantity float64 `json:"quantity,string"`
  LimitPrice float64 `json:"limit_price,string"`
  ExecutedQuantity float64 `json:"executed_quantity,string"`
  ExecutedPriceAvg float64 `json:"executed_price_avg,string"`
  Fee float64 `json:"fee,string"`
  Operations []*Operation `json:"operations"`
  CreatedTimestamp string `json:"created_timestamp"`
  UpdatedTimestamp string `json:"updated_timestamp"`
}

and this is how I'm trying to unmarshal it

func (limitOrder *LimitOrder) UnmarshalJSON(buf []byte) error {

  tmp := make(map[string]json.RawMessage)
  if err := json.Unmarshal(buf, &tmp); err != nil {
    return err
  }

  tmp2 := make(map[string]json.RawMessage)

  if err := json.Unmarshal(tmp["response_data"], &tmp2); err != nil {
    return err
  }

  if err := json.Unmarshal(tmp2["order"], limitOrder); err != nil {
    return err
  }

  return nil
}

[Nathan Kerr]

I figured it out.

First off, the posted playground had a different json string and did not use your UnmarshalJSON function. These made translating between the non-working setup described in your post and the working playground annoying. In the future, share the non-working setup.

At the point when I figured things out, my code was: https://play.golang.org/p/aMvz_JTrjD. This won't run on playground because it uses github.com/pkg/errors to add context to the errors so I could see which error was returned along with a much needed stack trace.

I found two problems with the implementation of UnmarshalJSON:

1. tmp["response_data"] and tmp2["order"] return zero values when the key is not found. This happened, first, because of the difference between the posted json and the json in the posted playground. Second, because of the second problem.

2. json.Unmarshal uses a type's UnmarshalJSON function if it exists to do the unmarshalling. This created a loop where *LimitOrder.UnmarshalJSON calls json.Unmarshal, which calls *LimitOrder.UnmarshalJSON, and so on. Line 71 prints out the call stack for the returned error that confirms this.

My recommended way of doing things is https://play.golang.org/p/kRKevuX8LW, that is write out the structs you need. This will also allow you to check the status_code from the response. If your LimitOrder will outlive the response then change ResponseData.LimitOrder to be a pointer.

Hope this helps.

[Diogo Ribeiro]

Thanks Nathan, it worked.

I didn't know that *LimitOrder.UnmarshalJSON would call json.Unmarshal.

[Nathan Kerr]

I assume you mean that you didn't know that json.Unmarshal would call *LimitOrder.UnmarshalJSON because *LimitOrder.UnmarshalJSON explicitly calls json.Unmarshal.

If so, I also had not realized the significance of the UnmarshalJSON name. This is one of the hazards of implicit interfaces.