Anyone here do anything with libpq?

[DFS]

https://www.postgresql.org/docs/9.6/static/libpq.html

Specifically, I'm curious about when/how often to clear a result set:

<quote>
PQclear
Frees the storage associated with a PGresult. Every command result
should be freed via PQclear when it is no longer needed.

void PQclear(PGresult *res);

You can keep a PGresult object around for as long as you need it; it
does not go away when you issue a new command, nor even if you close the
connection. To get rid of it, you must call PQclear. Failure to do this
will result in memory leaks in your application.
</quote>

'when it is no longer needed' sounds like it can be cleared once, at the
end of the script.

If you have experience with this, did you feel like it was OK to clear
it once at the end of the program, or every time before you used result
object?

[Rainer Weikusat]

DFS <nos...@dfs.com> writes:
> https://www.postgresql.org/docs/9.6/static/libpq.html

>
> <quote>
> PQclear
> Frees the storage associated with a PGresult. Every command result
> should be freed via PQclear when it is no longer needed.
>
> void PQclear(PGresult *res);
>
> You can keep a PGresult object around for as long as you need it; it
> does not go away when you issue a new command, nor even if you close
> the connection. To get rid of it, you must call PQclear. Failure to do
> this will result in memory leaks in your application.
> </quote>
>
> 'when it is no longer needed' sounds like it can be cleared once, at
> the end of the script.

There seems to be some kind of fundamental misunderstanding about the
nature of 'a result set' here. Specifically,

PGresult *res;

is a C pointer to a PGresult (structure) and in order to avoid memory
leaks, the result this pointer points to has to be freed before the
pointer is overwritten aka reused.

There's no need to free the result if the pointer won't be reused, IOW,
if the application is going to end next, anyway.

[Robert Klemme]

On 15.05.2017 16:41, DFS wrote:
> https://www.postgresql.org/docs/9.6/static/libpq.html
>
> Specifically, I'm curious about when/how often to clear a result set:
>
> <quote>
> PQclear
> Frees the storage associated with a PGresult. Every command result
> should be freed via PQclear when it is no longer needed.
>
> void PQclear(PGresult *res);
>
> You can keep a PGresult object around for as long as you need it; it
> does not go away when you issue a new command, nor even if you close the
> connection. To get rid of it, you must call PQclear. Failure to do this
> will result in memory leaks in your application.
> </quote>
>
> 'when it is no longer needed' sounds like it can be cleared once, at the
> end of the script.

What do you mean by "script"? Aren't we talking about C or C++ here?

> If you have experience with this, did you feel like it was OK to clear
> it once at the end of the program, or every time before you used result
> object?

My advice is to clear it as soon as you do not need it any more. This
might be less important for a short running program but if you are
writing some form of reusable code or a long running application then it
is clearly advisable to rid of the data as soon as it is processed. No
point in holding on to memory longer than it is needed.

Kind regards

robert

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

[DFS]

The question is, if I use a PGresult 10x during a program, should I
clear it between each use, or is one clear at the end OK?


In VB it's not necessary but I always closed them before the next usage.

Set rs = db.openrecordset("SELECT DATA;")
if rs.recordcount > 0 then
... do things
endif
rs.close

Set rs = db.openrecordset("SELECT DATA 2nd time;")
if rs.recordcount > 0 then
... do things
endif
rs.close

[DFS]

On 5/15/2017 1:54 PM, Robert Klemme wrote:
> On 15.05.2017 16:41, DFS wrote:
>> https://www.postgresql.org/docs/9.6/static/libpq.html
>>
>> Specifically, I'm curious about when/how often to clear a result set:
>>
>> <quote>
>> PQclear
>> Frees the storage associated with a PGresult. Every command result
>> should be freed via PQclear when it is no longer needed.
>>
>> void PQclear(PGresult *res);
>>
>> You can keep a PGresult object around for as long as you need it; it
>> does not go away when you issue a new command, nor even if you close the
>> connection. To get rid of it, you must call PQclear. Failure to do this
>> will result in memory leaks in your application.
>> </quote>
>>
>> 'when it is no longer needed' sounds like it can be cleared once, at the
>> end of the script.
>
> What do you mean by "script"? Aren't we talking about C or C++ here?

at the end of the C program, then.

I don't get hung up on scripting vs programming, compiled vs
interpreted, etc.

>> If you have experience with this, did you feel like it was OK to clear
>> it once at the end of the program, or every time before you used result
>> object?
>
> My advice is to clear it as soon as you do not need it any more. This
> might be less important for a short running program but if you are
> writing some form of reusable code or a long running application then it
> is clearly advisable to rid of the data as soon as it is processed. No
> point in holding on to memory longer than it is needed.
>
> Kind regards
>
> robert

Thanks

[Rainer Weikusat]

Well, you can't do this (as I already wrote): Queries return pointers to
result objects, not result objects. And if you overwrite a pointer
variable (type PGresult *) pointing to the last result with a pointer to
the next result, the old result will be leaked.

[Robert Klemme]

On 16.05.2017 20:01, DFS wrote:
> On 5/15/2017 1:54 PM, Robert Klemme wrote:

>> What do you mean by "script"? Aren't we talking about C or C++ here?
>
>
> at the end of the C program, then.
>
> I don't get hung up on scripting vs programming, compiled vs
> interpreted, etc.

What you call "get hung up" is trying to make sure what you wrote is
properly understood.

Cheers

[George Neuner]

On Tue, 16 May 2017 14:01:24 -0400, DFS <nos...@dfs.com> wrote:


>I don't get hung up on scripting vs programming, compiled vs
>interpreted, etc.

Well, you *should* get "hung up" on the differences between languages
because those differences affect using the languages correctly.


Many popular languages now are "managed" - i.e. the runtime
environment actively tries to help the programmer avoid mistakes.
Managed languages usually have automatic "garbage collection" which
frees memory used by objects when the program no longer has any
references (pointers) to them.

You cannot accidentally "lose" an object in a managed language because
you never "free" objects yourself - the runtime GC system watches what
the program does and knows when it is safe to free something.

C and C++ are not managed and they do not automatically free anything.
If you overwrite a pointer to an object, you lose the object if the
pointer was the only one referencing it.


>'when [PGResult object] is no longer needed' sounds like it can be

>cleared once, at the end of the script.

> :

>In VB it's not necessary but I always closed them before the next
>usage.
>
> Set rs = db.openrecordset("SELECT DATA;")
> if rs.recordcount > 0 then
> ... do things
> endif
> rs.close
>
> Set rs = db.openrecordset("SELECT DATA 2nd time;")
> if rs.recordcount > 0 then
> ... do things
> endif
> rs.close

VB has garbage collection. When you set 'rs', the object it
previously referred to is recycled. You could remove the close calls
entirely and the program probably[1] still would not leak memory.

C doesn't have garbage collection[2]. If you lose the result pointer
and don't call PGClear on it, the memory used by the result set is
lost.

There are times when you might want to keep results around for a
while. You might, for instance, pass the results to a separate thread
for processing. But then that thread has to call PGClear on the
pointer when it is done.


George

[1] depends on the 'db' object implementation.

[2] there are some automatic GC libraries for C and C++, which replace
the normal malloc/free, new/delete calls with managed versions. The
most well respected ones are the Boehm-Demers-Weiser collector and its
offshoot TinyGC.

http://www.hboehm.info/gc/
https://github.com/ivmai/tinygc

There are some others by Barlett, Detlefs, etc., including some
copying/moving collectors for C++, but they aren't as well known ...
and they place some (minor) restrictions on how you can code.

For some uses a GC library can work very well, but it isn't the same
as built-in GC where the compiler and runtime environment cooperate.