request context for “long running requests”

[Ladislav Thon]

Hi,

few days ago, we (me, Michal, Martin and Matej) discussed request context activation/propagation in the context of gRPC, but realized this has (or may have) broader impact. Hence we wanted to discuss this publicly.

What happens now in gRPC (but seemingly also in SSE, and maybe elsewhere too?) is that the request context is activated at the beginning of service call processing. The request context state is then captured, and activated again on each asynchronous callback pertaining to the same gRPC service call. Only when the gRPC call finishes is the request context destroyed.

This means that, for example, gRPC calls that use a database will each have their own Hibernate session, and that Hibernate session will be available for the entire duration of the gRPC call (because sessions are request-scoped, including Hibernate Reactive).

This sounds nice, but the thing with gRPC calls is that they can be streaming. Such stream may last for hours or days – and for sure you don’t want a session open for so long.

We could think of a few options how to tackle this:

- Keep doing what we’re doing. Works nice, until it doesn’t.
- Stop doing what we’re doing. Request context wouldn’t be propagated to the individual callbacks of a gRPC service call. Prevents sessions from living too long, at the cost of each part of the service call having a different session and hence a different transaction (which may lead to perceived data inconsistencies).
- Configurable behavior. It could be opt-in or opt-out, but either way, user would declare if the stream is long-running or not, which would then control how request context is activated/propagated.

Any thoughts? Other options?

Thanks,

LT

[Sanne Grinovero]

Thanks for raising this. Personally I don't love the fact that Hibernate's reactive sessions are bound to the "request context" by default - this has always been problematic for the blocking ORM in the past as well, and personally I've found it much more intuitive to manage scopes explicitly.

It's tempting to remove this assumption and recommend explicit boundaries for sake of avoiding much pain; however without the notion of "current Session" the Panache APIs don't look that sweet anymore - so while we might want to consider changing how the Session lifecycle is controlled, we need a notion of "current" being stored in the context. Ideally IMO it should match the transaction boundaries like we do for blocking ORM - and considering we made transactions required, that might be reasonable to explore?

Regarding having a Session open for days.. yea I don't generally recommend that either, but there might be reasonable need occasionally. Wouldn't this be one? To reconnect with the other topic, I'd say it depends on your transactionality expectations: are you planning to process the whole streaming call in the same transaction? Then I don't see why not, just ensure you do a little memory management by flushing & clearing the Session instance at appropriate points, as anyone needs to do when processing non trivial amounts of data: this need isn't really triggered by the long time of it staying open (even days) but by the amount of data. The length in terms of time however makes me worry about locking issues in the DB as we keep hold on critical resources, and needs for ACD out of ACID; if it really needs to take very long perhaps what you need is to decompose the operation in smaller transactions.

If we made behaviour configurable, I worry it gets both complex and still error-prone; would probably be better to enforce via API changes to only allow accessing sessions within the "transaction scope".

Thanks

--
You received this message because you are subscribed to the Google Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CALbocOmUBsVEkkZXdf-1wmjsgZyvRCCX_JnXTroO21PBxzocbA%40mail.gmail.com.

[Stuart Douglas]

On Tue, 22 Jun 2021 at 00:06, Ladislav Thon <lad...@gmail.com> wrote:

Hi,

few days ago, we (me, Michal, Martin and Matej) discussed request context activation/propagation in the context of gRPC, but realized this has (or may have) broader impact. Hence we wanted to discuss this publicly.

What happens now in gRPC (but seemingly also in SSE, and maybe elsewhere too?) is that the request context is activated at the beginning of service call processing. The request context state is then captured, and activated again on each asynchronous callback pertaining to the same gRPC service call. Only when the gRPC call finishes is the request context destroyed.

This means that, for example, gRPC calls that use a database will each have their own Hibernate session, and that Hibernate session will be available for the entire duration of the gRPC call (because sessions are request-scoped, including Hibernate Reactive).

Note that for blocking hibernate sessions are only request scoped if you are not using transactions, otherwise they are transaction scoped. The request scoped session will only be used if you attempt to use hibernate outside a transaction.

@Sanne Grinovero does a EntityManager that is not bound to a transaction tie up any database resources? I would assume it would just get a new connection per operation?

Stuart

 

This sounds nice, but the thing with gRPC calls is that they can be streaming. Such stream may last for hours or days – and for sure you don’t want a session open for so long.

We could think of a few options how to tackle this:

- Keep doing what we’re doing. Works nice, until it doesn’t.
- Stop doing what we’re doing. Request context wouldn’t be propagated to the individual callbacks of a gRPC service call. Prevents sessions from living too long, at the cost of each part of the service call having a different session and hence a different transaction (which may lead to perceived data inconsistencies).
- Configurable behavior. It could be opt-in or opt-out, but either way, user would declare if the stream is long-running or not, which would then control how request context is activated/propagated.

Any thoughts? Other options?

Thanks,

LT

--

[Sanne Grinovero]

On Tue, Jun 22, 2021 at 12:33 AM Stuart Douglas <sdou...@redhat.com> wrote:

On Tue, 22 Jun 2021 at 00:06, Ladislav Thon <lad...@gmail.com> wrote:

Hi,

few days ago, we (me, Michal, Martin and Matej) discussed request context activation/propagation in the context of gRPC, but realized this has (or may have) broader impact. Hence we wanted to discuss this publicly.

What happens now in gRPC (but seemingly also in SSE, and maybe elsewhere too?) is that the request context is activated at the beginning of service call processing. The request context state is then captured, and activated again on each asynchronous callback pertaining to the same gRPC service call. Only when the gRPC call finishes is the request context destroyed.

This means that, for example, gRPC calls that use a database will each have their own Hibernate session, and that Hibernate session will be available for the entire duration of the gRPC call (because sessions are request-scoped, including Hibernate Reactive).

Note that for blocking hibernate sessions are only request scoped if you are not using transactions, otherwise they are transaction scoped. The request scoped session will only be used if you attempt to use hibernate outside a transaction.

Right, and we don't allow using Hibernate ORM outside of a transaction. But there is an inconsistency here with Hibernate Reactive, which is what Ladislav is having problems with, as this one is not linked to JTA so it is indeed tied to the request scope.

 

@Sanne Grinovero does a EntityManager that is not bound to a transaction tie up any database resources? I would assume it would just get a new connection per operation?

We don't allow opening an EntityManager without a transaction; a blocking EntityManager will defer the acquisition of a connection up to the very last moment it needs one, but then will hold on to it until closing. This is different from earlier versions of Quarkus, which used a new connection per operation - for efficiency reasons it's better to reuse the same connection, except of course perhaps if you're keeping a Session/EntityManager open for days but we don't expect this to be common.

But Hibernate Reactive works differently; in this case a connection is acquired right away as it needs to start the transaction on the connection, then it needs to hold onto it as it can't move the transaction to a new connection.

Thanks,

Sanne

 

Stuart

 

This sounds nice, but the thing with gRPC calls is that they can be streaming. Such stream may last for hours or days – and for sure you don’t want a session open for so long.

We could think of a few options how to tackle this:

- Keep doing what we’re doing. Works nice, until it doesn’t.
- Stop doing what we’re doing. Request context wouldn’t be propagated to the individual callbacks of a gRPC service call. Prevents sessions from living too long, at the cost of each part of the service call having a different session and hence a different transaction (which may lead to perceived data inconsistencies).
- Configurable behavior. It could be opt-in or opt-out, but either way, user would declare if the stream is long-running or not, which would then control how request context is activated/propagated.

Any thoughts? Other options?

Thanks,

LT

--
You received this message because you are subscribed to the Google Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CALbocOmUBsVEkkZXdf-1wmjsgZyvRCCX_JnXTroO21PBxzocbA%40mail.gmail.com.

--

Sanne Grinovero

Architect, Hibernate team

Sr. Principal Software Engineer

Red Hat UK Ltd

[Stephane Epardaud]

What examples do you have of those long running gRPC calls? What do they do, for so long?

--

You received this message because you are subscribed to the Google Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CALbocOmUBsVEkkZXdf-1wmjsgZyvRCCX_JnXTroO21PBxzocbA%40mail.gmail.com.

--

Stéphane Épardaud

[Martin Kouba]

On 24. 06. 21 11:00, Stephane Epardaud wrote:
> What examples do you have of those long running gRPC calls? What do they
> do, for so long?

It's not "a single call" but a stream (possibly bi-directional) of
events... Imagine a client (e.g. a mobile app) connected to the server
and sending/receiving updates for the time the app is open.

>
> On Mon, 21 Jun 2021 at 16:06, Ladislav Thon <lad...@gmail.com
> <mailto:lad...@gmail.com>> wrote:
>
> Hi,
>
> few days ago, we (me, Michal, Martin and Matej) discussed request
> context activation/propagation in the context of gRPC, but realized
> this has (or may have) broader impact. Hence we wanted to discuss
> this publicly.
>
> What happens now in gRPC (but seemingly also in SSE, and maybe
> elsewhere too?) is that the request context is activated at the
> beginning of service call processing. The request context state is
> then captured, and activated again on each asynchronous callback
> pertaining to the same gRPC service call. Only when the gRPC call
> finishes is the request context destroyed.
>
> This means that, for example, gRPC calls that use a database will
> each have their own Hibernate session, and that Hibernate session
> will be available for the entire duration of the gRPC call (because
> sessions are request-scoped

> <https://github.com/quarkusio/quarkus/blob/main/extensions/hibernate-orm/runtime/src/main/java/io/quarkus/hibernate/orm/runtime/RequestScopedSessionHolder.java>,
> including Hibernate Reactive
> <https://github.com/quarkusio/quarkus/blob/main/extensions/hibernate-reactive/runtime/src/main/java/io/quarkus/hibernate/reactive/runtime/ReactiveSessionProducer.java>).

>
> This sounds nice, but the thing with gRPC calls is that they can be
> streaming. Such stream may last for hours or days – and for sure you
> don’t want a session open for so long.
>
> We could think of a few options how to tackle this:
>
> - Keep doing what we’re doing. Works nice, until it doesn’t.
> - Stop doing what we’re doing. Request context wouldn’t be
> propagated to the individual callbacks of a gRPC service call.
> Prevents sessions from living too long, at the cost of each part of
> the service call having a different session and hence a different
> transaction (which may lead to perceived data inconsistencies).
> - Configurable behavior. It could be opt-in or opt-out, but either
> way, user would declare if the stream is long-running or not, which
> would then control how request context is activated/propagated.
>
> Any thoughts? Other options?
>
> Thanks,
>
> LT
>
> --
> You received this message because you are subscribed to the Google
> Groups "Quarkus Development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to quarkus-dev...@googlegroups.com

> <mailto:quarkus-dev...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/quarkus-dev/CALbocOmUBsVEkkZXdf-1wmjsgZyvRCCX_JnXTroO21PBxzocbA%40mail.gmail.com

> <https://groups.google.com/d/msgid/quarkus-dev/CALbocOmUBsVEkkZXdf-1wmjsgZyvRCCX_JnXTroO21PBxzocbA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
>
>
> --
> Stéphane Épardaud

>
> --
> You received this message because you are subscribed to the Google
> Groups "Quarkus Development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to quarkus-dev...@googlegroups.com

> <mailto:quarkus-dev...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/quarkus-dev/CAKU9E9ttyXfysorfz_0ROJF7brqTUqXUnLFy4yRaAea-ciVysQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/quarkus-dev/CAKU9E9ttyXfysorfz_0ROJF7brqTUqXUnLFy4yRaAea-ciVysQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

--
Martin Kouba
Software Engineer
Red Hat, Czech Republic

[Ladislav Thon]

I have no concrete examples -- but quick googling shows that gRPC is used that way, and it's intended to be used that way (among others, of course). What seems to be one of the typical use cases is long-running notification subscriptions.

LT

čt 24. 6. 2021 v 11:00 odesílatel Stephane Epardaud <stephane...@gmail.com> napsal:

[Sanne Grinovero]

On Thu, 24 Jun 2021 at 10:26, Ladislav Thon <lad...@gmail.com> wrote:

I have no concrete examples -- but quick googling shows that gRPC is used that way, and it's intended to be used that way (among others, of course). What seems to be one of the typical use cases is long-running notification subscriptions.

When it comes to bridging an API to a database, I think it's our user's responsibility to decide how they want to break it up in small transactional units of work: they most likely don't want transactions running for days, which implies having to think about eventual consistency and compensating transactions; how this gets modelled often requires business-specific ad-hoc logic so we can't automate it all transparently, especially conflict resolutions and error handling.

What we can do is to help our users to define the boundaries of the smaller atomic transactions they need, and provide the APIs they will need to use within those boundaries.

This also relates to the reasons for which we're not exposing query results as "Multi", but Single(s) of List. The concept of long running streams doesn't map well on a transactional DB and requires some engineering. Luckily for us as, otherwise people wouldn't be using middleware but just deploy some automatic protocol converter ;) Let's not forget many databases do expose HTTP, JSON, XML and even gRPC natively.. but the devil is in the details.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CALbocOmFD6aZpC3aTepP1e3tgLkfQq-Ph45NYeZmf8V%2B9O%3DHsg%40mail.gmail.com.

[Erin Schnabel]

gRPC is multiplexed as IIOP was (or as websockets could be). Definitely not single request/single response.

A quick google:

A gRPC channel uses a single HTTP/2 connection, and concurrent calls are multiplexed on that connection. When the number of active calls reaches the connection stream limit, additional calls are queued in the client.

If you tentatively hurt your brain and think back to managing multiple requests over long-running IIOP connections...

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CALbocOmFD6aZpC3aTepP1e3tgLkfQq-Ph45NYeZmf8V%2B9O%3DHsg%40mail.gmail.com.

[Ladislav Thon]

čt 24. 6. 2021 v 12:54 odesílatel Sanne Grinovero <sa...@hibernate.org> napsal:

On Thu, 24 Jun 2021 at 10:26, Ladislav Thon <lad...@gmail.com> wrote:

I have no concrete examples -- but quick googling shows that gRPC is used that way, and it's intended to be used that way (among others, of course). What seems to be one of the typical use cases is long-running notification subscriptions.

When it comes to bridging an API to a database, I think it's our user's responsibility to decide how they want to break it up in small transactional units of work: they most likely don't want transactions running for days, which implies having to think about eventual consistency and compensating transactions; how this gets modelled often requires business-specific ad-hoc logic so we can't automate it all transparently, especially conflict resolutions and error handling.

What we can do is to help our users to define the boundaries of the smaller atomic transactions they need, and provide the APIs they will need to use within those boundaries.

Right, agree. My initial idea was that we could mark the gRPC service methods (and possibly other methods, SSE comes to mind again) as, say, `@LongRunningRequest` (better name would be good, as this looks dangerously close to the LRA specification). This would change how we activate the request context: instead of having a single request context for the entire duration of the stream, each "chunk" would get its own request context.

I'm of course far from expert in this area, and as you said, the devil is in the details, so... now really sure if that's enough, or if it's more of a false sense of security, or anything.

I did hope for this kind of feedback for sure, so keep it coming! :-)

LT

[Max Rydahl Andersen]

instead of marking it as longrunning request is it more the other way around
that grpc should consider each streaming interaction a request ?

not sure if that is doable but if not that then some other boundary mechanism; probably manually
as Sanne mentions is the way forward ?

/max

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CALbocOn6CptwQ7uTqa6JZqZyOw-MZtXVknHx-9VYZNw80W74oQ%40mail.gmail.com.

/max
https://xam.dk/about

[Stephane Epardaud]

Well, yeah, we could think of a way to mark this specially, via an annotation, and offer an API to do it too.

Even if we go via the API route only, we'll need an annotation to disable the request context at the very least, or to make sure it's not propagated and is terminates before the end of the Multi (or gRPC equivalent).

I do wonder how many people use this, though, because in the case of SSE, we barely ever have bug reports, so I assume they're just not used that much outside of demos.

--

Stéphane Épardaud

[Max Rydahl Andersen]

Well, yeah, we could think of a way to mark this specially, via an
annotation, and offer an API to do it too.
Even if we go via the API route only, we'll need an annotation to disable
the request context at the very least, or to make sure it's not propagated
and is terminates before the end of the Multi (or gRPC equivalent).

I do wonder how many people use this, though, because in the case of SSE,
we barely ever have bug reports, so I assume they're just not used that
much outside of demos.

I still remember many Hibernate cases where customers been running in production
and it worked great for them for long period of times but then on some weird days they lost data or
couldn't scale properly.

root cause: having put sessions into http session or even in a static block in code.

Everything is awesome until it isn't :)

Not saying we have the same exact situation here but I'm sure if there is an API there are
someone using it somehow in ways we don't think they should - and they don't even realise it.

/max

are request-scoped
https://github.com/quarkusio/quarkus/blob/main/extensions/hibernate-orm/runtime/src/main/java/io/quarkus/hibernate/orm/runtime/RequestScopedSessionHolder.java ,
including Hibernate Reactive
https://github.com/quarkusio/quarkus/blob/main/extensions/hibernate-reactive/runtime/src/main/java/io/quarkus/hibernate/reactive/runtime/ReactiveSessionProducer.java
).

This sounds nice, but the thing with gRPC calls is that they can be

streaming. Such stream may last for hours or days – and for sure you don’t
want a session open for so long.

We could think of a few options how to tackle this:

• Keep doing what we’re doing. Works nice, until it doesn’t.

• Stop doing what we’re doing. Request context wouldn’t be propagated

to the individual callbacks of a gRPC service call. Prevents sessions from
living too long, at the cost of each part of the service call having a
different session and hence a different transaction (which may lead to
perceived data inconsistencies).

• Configurable behavior. It could be opt-in or opt-out, but either

way, user would declare if the stream is long-running or not, which would
then control how request context is activated/propagated.

Any thoughts? Other options?

Thanks,

LT

--
You received this message because you are subscribed to the Google
Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/quarkus-dev/CALbocOmUBsVEkkZXdf-1wmjsgZyvRCCX_JnXTroO21PBxzocbA%40mail.gmail.com

https://groups.google.com/d/msgid/quarkus-dev/CALbocOmUBsVEkkZXdf-1wmjsgZyvRCCX_JnXTroO21PBxzocbA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

--
Stéphane Épardaud

--
You received this message because you are subscribed to the Google
Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/quarkus-dev/CALbocOmFD6aZpC3aTepP1e3tgLkfQq-Ph45NYeZmf8V%2B9O%3DHsg%40mail.gmail.com

https://groups.google.com/d/msgid/quarkus-dev/CALbocOmFD6aZpC3aTepP1e3tgLkfQq-Ph45NYeZmf8V%2B9O%3DHsg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

--

You received this message because you are subscribed to the Google Groups
"Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit

https://groups.google.com/d/msgid/quarkus-dev/CALbocOn6CptwQ7uTqa6JZqZyOw-MZtXVknHx-9VYZNw80W74oQ%40mail.gmail.com
https://groups.google.com/d/msgid/quarkus-dev/CALbocOn6CptwQ7uTqa6JZqZyOw-MZtXVknHx-9VYZNw80W74oQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

/max
https://xam.dk/about

--
Stéphane Épardaud

/max
https://xam.dk/about