Jackson XML: design/roadmap discussion for XSD-driven binding limitations + potential contributions

[Simon Cockx]

At REGnosys we are running into fundamental limitations of Jackson's support for XML. I would like to know whether these limitations are deliberate trade-offs, or changeable design decisions that could be fixed. Based on that we are considering whether we can either extend Jackson in our codebase, contribute to Jackson directly, or move away from Jackson if it doesn't fit at all.

First of all: why Jackson?

Saying that we just want to ingest XML based on an XSD is somewhat hand-wavy - the JAXB project exists exactly for that use case. So maybe the question is better stated: why not JAXB? In short: the XSD is not our source of truth, our domain specific language is.

At REGnosys we maintain the open-source Rune DSL, a language specifically designed for modelling processes in the financial industry. One important component of the language is ingestion: the process of reading serial data (JSON, XML, CSV, ...) in various financial standard formats and representing it in a uniform way in our DSL. Many of these formats are XML-based and formally defined as multiple XSD files, such as FpML. To support ingesting of these data standards, we use the following steps.

1. Transform the XSD into Rune types. (similar to how JAXB transforms XSD to Java classes)
2. Annotate the Rune types and fields with additional serialization information. (similar to what both Jackson and JAXB do/support)
3. From this Rune model, generate Java code with custom annotations.
4. Using a custom Jackson annotation processor, deserialize using a Jackson object mapper.

Note that steps 2 to 4 are independent of the exact serial format: we don't just support XML, we also support JSON and CSV, and want to stay extensible for any future formats. That is exactly the attractiveness of Jackson and where we loose interest in JAXB: Jackson's design principles align perfectly with this goal of agnostic deserialisation and serialisation.

Issues with Jackson XML

Most of our issues come down to the way bean properties are represented. Their identity is purely based on the local name of the property being deserialized, but doesn't take into account surrounding context such as ordering, namespaces, or representation (e.g., XML attribute versus XML element).

Examples of problems we run into:

1. Having XML elements and XML attributes with the same name is unsupported.
   Issue also described here: https://stackoverflow.com/q/47199799/3083982
   E.g., <foo id="my-id"><id>MyElementId</id></foo>
2. The @JsonUnwrapped annotation breaks some XML features. Fundamentally this is because it replaces the `FromXMLParser` instance with a `TokenBuffer`-based parser, which breaks assumptions for some XML related features. One example is described here: https://github.com/FasterXML/jackson-dataformat-xml/issues/762
3. Jackson does not support XSD substitution groups, i.e., having a single property with multiple potential names, depending on which a specific subtype deserializer is used. Turns out that this is not a fundamental issue: we have already extended Jackson to support it in the open-source Rune Common project. See issue ticket here: https://github.com/FasterXML/jackson-dataformat-xml/issues/679
4. Having XML elements with the same local name, but a different namespace, is unsupported. See long-standing issue ticket here: https://github.com/FasterXML/jackson-dataformat-xml/issues/65
5. Having XML elements with the same local name, but with a different order, is unsupported. I don't see a direct issue open for this, but it is related to this comment: https://github.com/FasterXML/jackson-dataformat-xml/issues/676#issuecomment-2438049500
   E.g., deserializing A1 and A2 to two distinct properties: <foo><a>A1</a><b/><a>A2</a></foo>

While we have ideas of how to approach this, I am definitely not saying we have a perfect solution in mind yet. We are mostly looking to answer the question if it is worth looking for a solution in the first place, or if this is just a fundamental limitation of Jackson.

I'm happy to discuss here, but if possible, I would also be very happy to jump on a call sometime to talk through this. Whatever works best.

Thanks in advance.

[Tatu Saloranta]

On Wed, Jan 28, 2026 at 8:31 AM Simon Cockx <simon...@regnosys.com> wrote:

At REGnosys we are running into fundamental limitations of Jackson's support for XML. I would like to know whether these limitations are deliberate trade-offs, or changeable design decisions that could be fixed. Based on that we are considering whether we can either extend Jackson in our codebase, contribute to Jackson directly, or move away from Jackson if it doesn't fit at all.

Hi! Yes, this makes sense. I am not sure what the ultimate answer is (it is obviously up to you), but I can try to address more specific questions/concerns.

 

First of all: why Jackson?

Saying that we just want to ingest XML based on an XSD is somewhat hand-wavy - the JAXB project exists exactly for that use case. So maybe the question is better stated: why not JAXB? In short: the XSD is not our source of truth, our domain specific language is.

At REGnosys we maintain the open-source Rune DSL, a language specifically designed for modelling processes in the financial industry. One important component of the language is ingestion: the process of reading serial data (JSON, XML, CSV, ...) in various financial standard formats and representing it in a uniform way in our DSL. Many of these formats are XML-based and formally defined as multiple XSD files, such as FpML. To support ingesting of these data standards, we use the following steps.

1. Transform the XSD into Rune types. (similar to how JAXB transforms XSD to Java classes)
2. Annotate the Rune types and fields with additional serialization information. (similar to what both Jackson and JAXB do/support)
3. From this Rune model, generate Java code with custom annotations.
4. Using a custom Jackson annotation processor, deserialize using a Jackson object mapper.

Note that steps 2 to 4 are independent of the exact serial format: we don't just support XML, we also support JSON and CSV, and want to stay extensible for any future formats. That is exactly the attractiveness of Jackson and where we loose 

interest in JAXB: Jackson's design principles align perfectly with this goal of agnostic deserialisation and serialisation.

Agreed. Thank you for explaining the background -- I think it does align with Jackson goals at high level.

 

Issues with Jackson XML

Most of our issues come down to the way bean properties are represented. Their identity is purely based on the local name of the property being deserialized, but doesn't take into account surrounding context such as ordering, namespaces, or representation (e.g., XML attribute versus XML element).

Right: XML is probably THE trickiest format for Jackson to support (of ~10 supported formats).

And most name mapping being namespace-unaware is problematic, and I'd have guessed number one problem.

So as you say, these are known, unsolved problems.

 

In a way you could say Jackson supports XML-specific aspects (namespaces, attribute-vs-element, ordering dependency) on serialization side but not well on deserialization -- on deserialization these aspects are essentially ignored.

Examples of problems we run into:

1. Having XML elements and XML attributes with the same name is unsupported.
   Issue also described here: https://stackoverflow.com/q/47199799/3083982
   E.g., <foo id="my-id"><id>MyElementId</id></foo>
2. The @JsonUnwrapped annotation breaks some XML features. Fundamentally this is because it replaces the `FromXMLParser` instance with a `TokenBuffer`-based parser, which breaks assumptions for some XML related features. One example is described here: https://github.com/FasterXML/jackson-dataformat-xml/issues/762
3. Jackson does not support XSD substitution groups, i.e., having a single property with multiple potential names, depending on which a specific subtype deserializer is used. Turns out that this is not a fundamental issue: we have already extended Jackson to support it in the open-source Rune Common project. See issue ticket here: https://github.com/FasterXML/jackson-dataformat-xml/issues/679
4. Having XML elements with the same local name, but a different namespace, is unsupported. See long-standing issue ticket here: https://github.com/FasterXML/jackson-dataformat-xml/issues/65
5. Having XML elements with the same local name, but with a different order, is unsupported. I don't see a direct issue open for this, but it is related to this comment: https://github.com/FasterXML/jackson-dataformat-xml/issues/676#issuecomment-2438049500
   E.g., deserializing A1 and A2 to two distinct properties: <foo><a>A1</a><b/><a>A2</a></foo>

While we have ideas of how to approach this, I am definitely not saying we have a perfect solution in mind yet. We are mostly looking to answer the question if it is worth looking for a solution in the first place, or if this is just a fundamental limitation of Jackson.

Of these, (4) could be supported if databind used full `PropertyName` (which has "simple" and "namespace" part), so conceptually that is achievable, but implementation would be quite involved.

Ideally there'd be no overhead for other formats, which would probably require more extensibility for XML backend to override handling (lookups).

(1) is sort of related but trickier: XML "attributeness" handling is contained with XML components, only used on serialization (I think).

(3) would be generally useful and ideally would be implemented -- not sure of all complexities due to "flattening" of layers Jackson otherwise adds. I think it is doable, but like all of these, non trivial.

For (2) some support was added to allow format-backends to substitute their own `TokenBuffer` subtypes, but that's as far as that goes. Buffering is also problematic for some @JsonCreator induced buffering wrt `Collection` deserialization.

(5) is probably the trickiest. I am not familiar with that yet, would need to dig deeper.

Currently there isn't a ton of progress towards any of these (esp. as all are hard problems).

But there are no fundamental blockers, I think. This is probably bit awkward wrt defining which path to take.

I am happy to try to help in addressing these, for what that is worth.

 

I'm happy to discuss here, but if possible, I would also be very happy to jump on a call sometime to talk through this. Whatever works best.

I think discussing this here is good -- I will be out until next week now but wanted to send a quick response before that.

 

Alternatively Github Discussions on https://github.com/FasterXML/jackson-dataformat-xml/discussions would also work.

Thanks in advance.

Thank you,

-+ Tatu +-

 

--
You received this message because you are subscribed to the Google Groups "jackson-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jackson-dev...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/jackson-dev/474eea22-e935-4386-b2f3-1f1adfe65d06n%40googlegroups.com.

[Simon Cockx]

Thanks for the quick response Tatu! I am delighted that at least it is not an immediate "this will not work" conclusion because of fundamental design principles.

 

I think discussing this here is good -- I will be out until next week now but wanted to send a quick response before that.

I appreciate your time - no rush at all.

Out of curiosity, is any work related to these issues already on the Jackson roadmap, which we can piggyback off, or is there no concrete work planned in the area?

Just to zoom in a bit on (5), because you mention it is probably the trickiest, and it might be a good indication of "how far" we can go with Jackson. The use case I have described (deserialize two properties with the same name with a different order), is actually not an important use case on its own, but it becomes much more relevant in interaction with (2) (unwrapping) and (3) (substitution groups). Two use cases I have seen while POC-ing support for some real XSD's are described below.

a) Having the same property name on different levels in the Java pojo, but because of unwrapping they overlap.

Example structure taken straight out of a real XSD, but simplified.

Interpretation: you either have an `issuer` element followed by a single `tradeId` element, OR you have a `partyReference` element followed by a variable number of `tradeId` elements.

```

<xs:complexType name="Trade">
  <xs:choice>
  <xs:sequence>
<xs:element name="issuer" type="IssuerId"/>
<xs:element name="tradeId" type="TradeId"/>
</xs:sequence>
  <xs:sequence>

      <xs:element name="partyReference" type="PartyReference"/>
    <xs:element name="tradeId" type="TradeId" minOccurs="0" maxOccurs="unbounded"/>
  </xs:sequence>
  </xs:choice>
</xs:complexType>

```

We currently represent this something like the following in Java: (using records to concisely show structure - we actually use classes)

```

record Trade(TradeOpt1 opt1, TradeOpt2 opt2) {}

record TradeOpt1(IssuerId issuer, TradeId tradeId) {}

record TradeOpt2(PartyReference partyReference, List<TradeId> tradeIds) {}

```

where we unwrap `TradeOpt1` and `TradeOpt2`. At this point, however, when we encounter a `tradeId` element, we somehow need to know whether to set it to `TradeOpt1` or to add it to the list of `TradeOpt2`. Right now, BOTH happen. (in other situations I have seen one of the two taking precedence, depending on the exact unwrapping structure)

b) A substituted name overlaps with an already existing element name on the type

Another example structure based on what I have seen in a real XSD.

Note that the element called `substituted` can be substituted by an element called `foo`. 

```

<xs:complexType name="Root">
  <xs:sequence>
    <xs:element ref="substituted"/>
    <xs:element name="inbetween" type="xs:string"/>
    <xs:element name="foo" type="Foo"/>
  </xs:sequence>
</xs:complexType>

<xs:element name="substituted" type="Parent"/>
<xs:element name="foo" type="Foo" substitutionGroup="substituted"/>

<!-- assume type Foo extends type Parent -->

```

In this scenario, a sample such as
```
<root>
  <foo></foo>
  <inbetween>value</inbetween>
  <foo></foo>
</root>
```

should be able to decide that the first `foo` element should deserialize into the `substituted` property, and the second `foo` element should deserialize into the `foo` element, given below structure.

```

record Root(Parent substituted, String inbetween, Foo foo) {}

```

Thoughts...

In order to support this, I think it would require work to extend how Jackson is able to identify properties. Some ideas:

- based on element index, although that does not work well if some elements are optional, or if some elements can occur multiple times.

- based on a selector which allows relative matching, e.g., "the element that comes after another element", such as XPath.

... or a drastically different approach, e.g., deserializing using recursive descent with backtracking, instead of based on property names.

Then there is thinking about how to support this without breaking other backends. Again high-level ideas I can think of:

- making matching on `PropertyName` more generic. E.g., instead of fetching a deserializer straight from a map, add a layer of abstraction that exposes a method `findMatchingProperty`, which backends can override based on their own element identification. The default implementation would lookup a property in a map using `PropertyName`.

- entirely skipping the regular Jackson way of building deserializers, and creating a custom BeanDeserializer that implements its own lookup system.

- entirely skipping the regular Jackson way of building deserializers, and creating a custom recursive descent deserializer.

All of them seem like quite a chunk of work, and require careful thought about their implications. So: any thoughts on whether this is achievable at all? Other ideas?

I assume use cases (1) - (4) would be less involved than this, but as I show in my examples, they will break when they interact with (5), hence why I just want to check upfront whether (5) is doable at all.