portals, prepared statements and memos

[Andrei Matei]

Happy New Year friends!

I want to see if I can stimulate someone to do something around the state we maintain for portals. At the moment, a portal maintains a reference to the prepared statement whence it came. In turn, the prepared contains a Memo, which is used to produce a plan for the portal at a later time. I am guessing we maintain the whole memo on the prep stmt instead of a plan because we can't actually create a plan until the arguments for the statement are known*.
What bothers me is that portals keep references, indirectly, to the memo. When a portal is created (through a pgwire Bind command), the arguments are knows, so I think it'd be cool if the portal was created with the plan directly, rather than only creating the plan at Exec time.

Does this make sense? Any takers for a peer ack?

The specific reason why I bring this up is memory management. The long story is that I've been trying to revamp crdb's understanding of portals, which is very naive at the moment, and so I've been going down some rabbit holes. One is around the lifetime of portals - at the moment in crdb they're tied to the lifetime of the corresponding prepared statement, but they shouldn't be (i.e. in Postgres a portal can outlive the original prepared statement being closed). A problem I've run into when trying to rectify this is around the memory accounting of the things - prepared statements have some memory accounting, and portals hold references to (and share) prepared statements, and so if the lifetimes of the portals now becomes divorced from the prepared statement, I'd need to introduce reference counting to not actually consider the prepared stmt freed too early.

My first instinct was to rip out the accounting for prepared statements, as historically I've thought it was worse than useless, but now I see this memo attached to them, which a) is probably significant and b) has some sort of memory estimates going on.

Cheers!

- a_m

*) the pg docs have this interesting snippet: If the prepared statement has no parameters, or is executed repeatedly, the server might save the created plan and re-use it during subsequent Bind messages for the same prepared statement. However, it will do so only if it finds that a generic plan can be created that is not much less efficient than a plan that depends on the specific parameter values supplied.

[Radu Berinde]

I think getting the plan during Bind makes sense in principle. I can look into that.

[Nathan VanBenschoten]

Hey Andrei, if you're actively looking into reworking the lifetime of portals and their relation to that of prepared statements, please keep this issue in mind.

--
You received this message because you are subscribed to the Google Groups "CockroachDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cockroach-db...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cockroach-db/CAEthiYCkkTE2o0JB%3DYDS8pNjNqSobdv%2BaoTcczMqs%3DRv-H7KSw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Andrei Matei]

I think getting the plan during Bind makes sense in principle. I can look into that.

Thanks! 

[Raphael 'kena' Poss]

On 02-01-19 14:04, Radu Berinde wrote:
> I think getting the plan during Bind makes sense in principle. I can
> look into that.

As discussed in the other head, I recommend to not send a PR to let
plans linger unless you also have implemented memory accounting for them.


--
Raphael 'kena' Poss