Array uniqueItems not able to describe object items with specific keys.

[Pavel Zářecký]

Let's consider following JSON data example:

[
  {
    "name": "John",
    "surname": "Smith",
    "lastCheck": "Tue 23rd August 2016"
  },
  {
    "name": "John",
    "surname": "Doe",
    "lastCheck": "Wed 24th August 2016"
  },
  {
    "name": "Mary",
    "surname": "Jones",
    "lastCheck": "Tue 23rd August 2016"
  }
]

Now let's imagine that I'd provide following schema:

{
  "type": "array",
  "items": { "$ref": "#/definitions/PersonEntry" },
  "uniqueItems": true,
  "definitions": {
    "PersonEntry": {
      "type": "object",
      "required": ["name", "surname", "lastCheck"],
      "properties": {
        "name": { "type": "string" },
        "surname": { "type": "string" },
        "lastCheck": { "type": "string" }
      }
    }
  }
}

However with such a schema it would be successfully validated even this:

[
  {
    "name": "John",
    "surname": "Smith",
    "lastCheck": "Tue 23rd August 2016"
  },
  {
    "name": "John",
    "surname": "Smith",
    "lastCheck": "Wed 24th August 2016"
  }
]

... where same person appears twice in the list.

I would like to describe that the objects have some unique (primary) keys (name and surname in this example) based on which the uniqueness should be checked.

[Henry Andrews]

Hi Pavel,

You could split this out into two complementary array specifications, one to declare the uniqueness constraint that you want (on name and surname) and the other to add in the non-unique properties.

This schema says "there should be an array, which should validate both as an array of unique PersonNames, and as an array of not-necessarily-unique PersonChecks."  Scroll down below this for a bit more commentary on what's going on here:

{
  "allOf": [
    {
      "type": "array",
      "items": { "$ref": "#/definitions/PersonName" },
      "uniqueItems": true
    },
    {
      "type": "array",
      "items": { "$ref": "#/definitions/PersonCheck" }
    }
  ],
  "definitions": {
    "PersonName": {

      "type": "object",
      "required": ["name", "surname"],

      "properties": {
        "name": { "type": "string" },
        "surname": { "type": "string" }
      }
    },

    "PersonCheck": {
      "type": "object",
      "required": ["lastCheck"],
      "properties": {
        "lastCheck": { "type": "string" }
      }
    }
  }
}

From the point of view of validating each *individual* item in the array, the above schema is the same as (leaving out "definitions" this time):

{
  "type": "array",
  "items": {
    "allOf": [
      {"$ref": "#/definitions/PersonName" },
      { "$ref": "#/definitions/PersonCheck" }
    ]
  }
}

The difference with pulling the "allOf" up outside of the array schemas is that that allows you to specify uniqueItems: true on only one branch of the allOf.  You need each branch to separately define the validation for the entire array to do that.

thanks,

-henry

[Pavel Zářecký]

Perfect Henry, this didn't come to my mind. Thank you.

[Ben Hutton (@Relequestual)]

Henry, that's awesome! Good thinking!

cc Austin: Have you ever done something similar? This is a great example of how using $ref and definitions can be powerful. (Although it COULD be done without using ref, but the seperation really helps in terms of readability, at least for me.)


Ben

[Przemysław Wesołek]

Hi Henry,

Although I understand the intention, and would very much like this to be true, I don't see anything in a spec that would support your claim of "uniquness based on schema". Instead, what I see is "uniquness based on data" on wiki[1]:

([...] Definition of equality [...] for objects [...])

• have the same set of members (keys), and
• member values of both objects, for the same member, are equal according to this definition.

or "uniquness is axiomatic" in a spec[2]:

If this keyword has boolean value false, the instance validates successfully.  If it has boolean value true, the instance validates successfully if all of its elements are unique.

Are you sure that the tools interpret the equality limiting the concept to the properties of the object that are being validated, i.e. those from "properties" only? That would be nice, although it needs clarification/modification in v6 and on the wiki, as well as more thought for deeper objects validation (esp. for allOf/anyOf/oneOf validations etc.).

[1] https://github.com/json-schema/json-schema/wiki/uniqueItems
[2] https://tools.ietf.org/html/draft-wright-json-schema-validation-00#section-5.12

[Henry Andrews]

Hi Przemysław,

  I'm afraid I don't follow your concern.  The uniqueItems keyword is about value equality.  The way I split the schemas up ensures that the value equality constraint only applies to the combination of the name and surname fields, and not the entire object.  In other words, the lastCheck field does not affect uniqueness, which is what we want.  Otherwise, we could have the same person in the list twice simply because lastCheck is set to something different each time.

thanks,

-henry

[Przemysław Wesołek]

My concern is that you conclusion that only the properties you specify in the schema are taken account in equality checks, is not valid.

I took your schema, i.e. the one with check

{
    "type": "array",
    "items": { "$ref": "#/definitions/PersonName" },
    "uniqueItems": true
}

and run against the document that the OP wanted to quality as non valid:

[
  { "name": "John", "surname": "Smith", "lastCheck": "Tue 23rd August 2016" },

  { "name": "John", "surname": "Smith", "lastCheck": "Wed 24th August 2016" }
]

Both ajv[1] and fge-based online validator[2] claim it is valid. This is because (IMO) the equality check is based on the values in JSON, not the schema predicates that perform checks on a subset of properties.

[1] http://epoberezkin.github.io/ajv/
[2] http://json-schema-validator.herokuapp.com/

[Przemysław Wesołek]

W dniu poniedziałek, 24 października 2016 14:34:00 UTC+2 użytkownik Przemysław Wesołek napisał:

Hi Henry,

Although I understand the intention, and would very much like this to be true, I don't see anything in a spec that would support your claim of "uniquness based on schema". Instead, what I see is "uniquness based on data" on wiki[1]:

([...] Definition of equality [...] for objects [...])

• have the same set of members (keys), and
• member values of both objects, for the same member, are equal according to this definition.

or "uniquness is axiomatic" in a spec[2]:

If this keyword has boolean value false, the instance validates successfully.  If it has boolean value true, the instance validates successfully if all of its elements are unique.

My bad, didn't notice JSON Schema core defines objects equality, basically in the same way as the wiki does, in section 3.6 (v4) or 4.3 (v5).

Przemek

[Henry Andrews]

Hmm... interesting.  I see now.  So uniqueItems applies to the values in general, not just the values matched by the schema.  Actually that does make a certain amount of sense :-(

It would be useful to have a "unique in terms of the things that match this schema" concept, but I'm not quite sure how that would be implemented.  This does show that the spirit of "uniqueItems" can easily be circumvented by tacking some other property onto the objects within the array.

thanks,

-henry

---

From: Przemysław Wesołek <przemysla...@gmail.com>
To: JSON Schema <json-...@googlegroups.com>
Sent: Tuesday, October 25, 2016 5:03 AM
Subject: [json-schema] Re: Array uniqueItems not able to describe object items with specific keys.

--
You received this message because you are subscribed to the Google Groups "JSON Schema" group.
To unsubscribe from this group and stop receiving emails from it, send an email to json-schema...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.