xml namespaces - encoding ok, decoding result not ok

[Marcin Jurczuk]

Hi gophers,

I stuck with something that supposed to be easy - XML decoding. Encoding works perfeclty.

I'm trying for past two days to find working code for parsing SOAP message.

I have structures that are encoded correctly but decoding responses (same structure, just filled with values) is failing (sic!)

Here is sample code that encodes struct and then tries decode the same struct

https://play.golang.org/p/hE7vcXbymg

It seems that Go have a lot of issues with XML jsut small portion:

https://github.com/golang/go/issues/6800 (this one looks like my problem - not fixed since 2013 !)

https://github.com/golang/go/issues/9775

https://github.com/golang/go/issues/9519

Please help me to diagnose is it my problem or one of Go bugs

[roger peppe]

Namespaces are space-separated from tag names in the struct tags.

Changing that seems to make your example work OK:

https://play.golang.org/p/GBvhcWLew_

From the encoding/xml docs:

If the XMLName field has an associated tag of the form
"name" or "namespace-URL name", the XML element must have
the given name (and, optionally, name space) or else Unmarshal
returns an error.

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Marcin Jurczuk]

Your example is returning xml that is wrong from system I'm talking to.

My SOAP service requires tags in few different namespaces.

My code returns:

  <os:QUERYResponse>

Your code returns:

<QUERYResponse xmlns="os">

etc.

Such syntax is not ok. My Service is returning error. With my syntax - I'm getting responses.

Of course os namespace is declared at the begining of file:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:os="os.xsd.v1">

I looks like go have bug with Unmarshaling tags with namespaces. Bug exists at least 2 years, maybe more.

[roger peppe]

The sample XML in your code doesn't appear to be valid - it doesn't
actually define the namespace that the os namespace prefix refers to.

I am fully aware that encoding/xml has bugs, but I'm not sure
that this is one of them.

[Konstantin Khomoutov]

On Tue, Aug 01, 2017 at 10:48:23AM -0700, Marcin Jurczuk wrote:

> Your example is returning xml that is wrong from system I'm talking to.
> My SOAP service requires tags in few different namespaces.
> My code returns:
>

> * <os:QUERYResponse>*
>
> Your code returns:
>
> *<QUERYResponse xmlns="os">*
[...]
> > > https://play.golang.org/p/hE7vcXbymg
[...]

I'm with roger on this: in your example, your sample XML document's text
begins with

<?xml version="1.0" encoding="UTF-8"?>
<os:QUERYResponse>
...

which does not define that "os:" bit to denote an XML namespace prefix
anywhere. This means the parser does not treat it as such -- it merely
considers it to be an integral part of the so-called "local names" of
the elements in that XML document.

To add to the confusion, you have used those prefixes verbating in the
tags of the fields of the types intended to work with the corresponding
XML elements. That's why the parser worked: you told it to, say, parse
an element literally named "os:QUERYResponse" and so it did that.

If you really need to interpret that "os:" bit as an XML namespace
prefix, you have to do two things:

* Have that prefix defined in your XML document.
* Use the full namespace name when you refer to the names of your XML
elements - separated by a single space character from their local
parts.

Hence, say, your XML document should go like


<?xml version="1.0" encoding="UTF-8"?>
<os:QUERYResponse xmlns:os="http://foo.bar/baz">
...

and you use the full namespace name in your tags like in

type QueryResponse struct {
XMLName xml.Name `xml:"http://foo.bar/baz QUERYResponse"`

[Marcin Jurczuk]

Here is fully working/not working/not working example with whole SOAP envelope I'm getting from server:

If I use different structs for encoding/decoding it works.

https://play.golang.org/p/hVzB0lh2WH

Change line:

rx := new(SOAPEnvelope2)

to:

rx := new(SOAPEnvelope)

And you will get error that shouldn't exists since serialization suppose to be symmetric.

data varibable is copy/paste Encode output from .Encode()

I have exact the same problem like here:

https://github.com/golang/go/issues/9519

When I use different struct for Unmarshaling and Marshalling it works.

But this is a lot of boilerplate code and IMO serialization/deserialization to xml for the same struct should give exactly the same outuput :)

Your solution gives different output.

[roger peppe]

It seems to me like your unmarshaling isn't actually working
(look inside the decoded data).

Here's an alternative version:
https://play.golang.org/p/CAChEa4-Kp

It's still not exactly right, as the DID tag is in the empty namespace
in the original XML, but is in os.xsd.v1 in the encoded version.
Does that cause your decoders to fail?
That's a bug in the encoding/xml package, as I don't think it's
possible to explicitly specify the empty namespace in a struct tag.

In your code, I believe that struct tags like `xml:"os:QUERYResponse"`
are only working by accident, because encoding/xml isn't interpreting
it as a namespace-prefixed tag name but instead just treating it as
a whole tag, so just printing it as is. That's probably why unmarshaling
isn't working.

BTW, I'd suggest trying a fork of encoding/xml that gets namespaces
somewhat more right: github.com/juju/xml. It was in the standard library
at some point, and hopefully will get there again when I have some
energy to do it.

cheers,
rog.