Happy New Year friends!
I want to see if I can stimulate someone to do something around the state we maintain for portals. At the moment, a portal maintains a reference to the prepared statement whence it came. In turn, the prepared contains a Memo
, which is used to produce a plan for the portal at a later time. I am guessing we maintain the whole memo on the prep stmt instead of a plan because we can't actually create a plan until the arguments for the statement are known*.
What bothers me is that portals keep references, indirectly, to the memo. When a portal is created (through a pgwire Bind command), the arguments are knows, so I think it'd be cool if the portal was created with the plan directly, rather than only creating the plan at Exec time.
Does this make sense? Any takers for a peer ack?
The specific reason why I bring this up is memory management. The long story is that I've been trying to revamp crdb's understanding of portals, which is very naive at the moment, and so I've been going down some rabbit holes. One is around the lifetime of portals - at the moment in crdb they're tied to the lifetime of the corresponding prepared statement, but they shouldn't be (i.e. in Postgres a portal can outlive the original prepared statement being closed). A problem I've run into when trying to rectify this is around the memory accounting of the things - prepared statements have some memory accounting, and portals hold references to (and share) prepared statements, and so if the lifetimes of the portals now becomes divorced from the prepared statement, I'd need to introduce reference counting to not actually consider the prepared stmt freed too early.
My first instinct was to rip out the accounting for prepared statements, as historically I've thought it was worse than useless, but now I see this memo attached to them, which a) is probably significant and b) has some sort of memory estimates going on.
*) the pg docs have this interesting snippet: If the prepared statement has no parameters, or is executed repeatedly, the server might save the created plan and re-use it during subsequent Bind messages for the same prepared statement. However, it will do so only if it finds that a generic plan can be created that is not much less efficient than a plan that depends on the specific parameter values supplied.