Let FilteringParserDelegate fail when a matching property does not exist

[Dai MIKURUBE]

Hi,

I have a sequence of multiple JSONs in a stream, and wanted to filter the sequence by FilteringParserDelegate / JsonPointerBasedFilter.

It works in normal matching cases. But, I wanted to raise an error when a matching property does not exist.

For example, in the following case :

        JsonParser parser = new FilteringParserDelegate(
                factory.createParser("{\"foo\":{\"bar\":\"baz\"}}{\"xxx\":{\"yyy\":\"zzz\"}}{\"foo\":{\"bar\":\"qux\"}}"),
                new JsonPointerBasedFilter("/foo"),
                TokenFilter.Inclusion.ONLY_INCLUDE_ALL,
                true
                );

This |parser| returns {"bar":"baz"}, and {"bar":"qux"}, with just skipping {"xxx":{"yyy":"zzz"}}.

Here, I wanted to raise an error (Exception) while reading {"xxx":{"yyy":"zzz"}}.

Does anyone know how to realize it?  I'm ready to write my own TokenFilter, or to expand JsonPointerBasedFilter by myself.

Thanks.

[Tatu Saloranta]

On Tue, Sep 19, 2023 at 10:31 AM Dai MIKURUBE <dmik...@acm.org> wrote:
>
> Hi,
>
> I have a sequence of multiple JSONs in a stream, and wanted to filter the sequence by FilteringParserDelegate / JsonPointerBasedFilter.
>
> It works in normal matching cases. But, I wanted to raise an error when a matching property does not exist.

Sounds reasonable.

>
> For example, in the following case :
>
> JsonParser parser = new FilteringParserDelegate(
> factory.createParser("{\"foo\":{\"bar\":\"baz\"}}{\"xxx\":{\"yyy\":\"zzz\"}}{\"foo\":{\"bar\":\"qux\"}}"),
> new JsonPointerBasedFilter("/foo"),
> TokenFilter.Inclusion.ONLY_INCLUDE_ALL,
> true
> );
>
> This |parser| returns {"bar":"baz"}, and {"bar":"qux"}, with just skipping {"xxx":{"yyy":"zzz"}}.
>
> Here, I wanted to raise an error (Exception) while reading {"xxx":{"yyy":"zzz"}}.
>
>
> Does anyone know how to realize it? I'm ready to write my own TokenFilter, or to expand JsonPointerBasedFilter by myself.

I haven't done this, but looking at `JsonPointerBasedFilter` I would
probably try overriding `includeProperty()` method to catch the case
of non-match. Note that you'd also need to override `includeElement`,
and for both make sure to construct an instance of your own sub-class.
I think that should allow you to do what you want.
I'll also file an issue for minor improvement for
`JsonPointerBasedFilter` to make sub-classing bit safer.

-+ Tatu +-


>
>
> Thanks.
>
> --
> You received this message because you are subscribed to the Google Groups "jackson-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jackson-user...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/6acc6b25-7292-4b31-a08f-8f541a075158n%40googlegroups.com.

[Dai MIKURUBE]

Thanks, Tatu!

> I haven't done this, but looking at `JsonPointerBasedFilter` I would
> probably try overriding `includeProperty()` method to catch the case
> of non-match. Note that you'd also need to override `includeElement`,
> and for both make sure to construct an instance of your own sub-class.
> I think that should allow you to do what you want.
> I'll also file an issue for minor improvement for
> `JsonPointerBasedFilter` to make sub-classing bit safer.

But I'm afraid that, IIUC, `includeProperty()` works "reactively" against the

actual incoming JSON stream. Then, it works for matching properties, but

it could not work for non-match. (In the example, `includeProperty()` might

never be called against "foo" while reading {"xxx":{"yyy":"zzz"}}.)

`TokenFilter` has many other callback methods. I wondered if some of them

could be used, but not dived deeper yet. Or, otherwise, I thought that I might

have to hack `FilteringParserDelegate` reimplemented.

2023年9月20日(水) 14:09 Tatu Saloranta <ta...@fasterxml.com>:

You received this message because you are subscribed to a topic in the Google Groups "jackson-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jackson-user/bLZTjjjXZ28/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jackson-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/CAL4a10jR5MgpgF8YBzG%2BRZMZFv9%2Br_WCqgCadNERRTaNH7QiDw%40mail.gmail.com.

--

Dai MIKURUBE
   dmik...@acm.org

[Dai MIKURUBE]

> But I'm afraid that, IIUC, `includeProperty()` works "reactively" against the
> actual incoming JSON stream. Then, it works for matching properties, but
> it could not work for non-match. (In the example, `includeProperty()` might
> never be called against "foo" while reading {"xxx":{"yyy":"zzz"}}.)

Ah..., wait, if :

1) `TokenFilter` (extended `JsonPointerBasedFilter`) can be stateful, and

2) `TokenFilter` (extended `JsonPointerBasedFilter`) can detect the end of

   the object or the array ("}" or "]"),

it may probably be feasible by :

1') changing the state when `includeProperty` receives "foo", and

2') raising an Exception when the state is not changed at "}"

Let me think a little bit more.

I hope (1) is okay. (2) may be feasible by implementing `filterFinishObject`?

2023年9月20日(水) 14:30 Dai MIKURUBE <dmik...@acm.org>:

--

Dai MIKURUBE
   dmik...@acm.org

[Dai MIKURUBE]

Hi,

While I was tackling this, I found that `filterFinishObject` is not called properly under some conditions.

Then, I created my first pull request for jackson-core :

https://github.com/FasterXML/jackson-core/pull/1111

2023年9月21日(木) 0:43 Dai MIKURUBE <dmik...@acm.org>:

--

Dai MIKURUBE
   dmik...@acm.org

[Tatu Saloranta]

On Tue, Sep 19, 2023 at 10:30 PM Dai MIKURUBE <dmik...@acm.org> wrote:
>
> Thanks, Tatu!
>
> > I haven't done this, but looking at `JsonPointerBasedFilter` I would
> > probably try overriding `includeProperty()` method to catch the case
> > of non-match. Note that you'd also need to override `includeElement`,
> > and for both make sure to construct an instance of your own sub-class.
> > I think that should allow you to do what you want.
> > I'll also file an issue for minor improvement for
> > `JsonPointerBasedFilter` to make sub-classing bit safer.
>
> But I'm afraid that, IIUC, `includeProperty()` works "reactively" against the
> actual incoming JSON stream. Then, it works for matching properties, but
> it could not work for non-match. (In the example, `includeProperty()` might
> never be called against "foo" while reading {"xxx":{"yyy":"zzz"}}.)

Ah. My bad, I misread your question to think you wanted to report
unwanted (extra) properties.

True, there is nothing to indicate absence; that must be derived from state.

>
> `TokenFilter` has many other callback methods. I wondered if some of them
> could be used, but not dived deeper yet. Or, otherwise, I thought that I might
> have to hack `FilteringParserDelegate` reimplemented.

Right, I think `TokenFilter.filterFinishObject()` and checking nesting
allows you to detect absence of content (assuming you keep track etc,
nothing automated).

-+ Tatu +-

> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/CAJ3Zeojs-NxpcaL0Qu%2B4Bq9%2BzB0gHXr544K48vF3bMtk0JnC1A%40mail.gmail.com.

[Tatu Saloranta]

On Wed, Sep 20, 2023 at 8:43 AM Dai MIKURUBE <dmik...@acm.org> wrote:
>
>
> > But I'm afraid that, IIUC, `includeProperty()` works "reactively" against the
> > actual incoming JSON stream. Then, it works for matching properties, but
> > it could not work for non-match. (In the example, `includeProperty()` might
> > never be called against "foo" while reading {"xxx":{"yyy":"zzz"}}.)
>
> Ah..., wait, if :
>
> 1) `TokenFilter` (extended `JsonPointerBasedFilter`) can be stateful, and
> 2) `TokenFilter` (extended `JsonPointerBasedFilter`) can detect the end of
> the object or the array ("}" or "]"),
>
> it may probably be feasible by :
>
> 1') changing the state when `includeProperty` receives "foo", and
> 2') raising an Exception when the state is not changed at "}"
>
> Let me think a little bit more.
> I hope (1) is okay. (2) may be feasible by implementing `filterFinishObject`?

Yes, exactly.

Minus that bug you found... :-/

-+ Tatu +-

> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/CAJ3Zeoh7ixM%3DdnkFnT%2B91HgZZu-xgKkACyWskSLeANqAtR4PHQ%40mail.gmail.com.