Firebird 5.0.3

[Tomasz Dubiel]

Hello.

What can be the reason of growing RAM no matter what architecture? Ubuntu 24.04.3

Database is only 1 GB, daily usage can grow even to 40 GB and use all available RAM .

Something is wrong, we try to find what.

It grows on average 1GB per hour.

There is about 100 connections.

Best regards,

Tomek.

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 11.02.2026 12:39:
> What can be the reason of growing RAM no matter what architecture? Ubuntu 24.04.3
> Database is only 1 GB, daily usage can grow even to 40 GB and use all available
> RAM .
> Something is wrong, we try to find what.
> It grows on average 1GB per hour.

What do you see in MON$MEMORY_USAGE?

--
WBR, SD.

[Köditz, Martin]

Guten Tag und vielen Dank für Ihre Nachricht.

Leider bin ich zurzeit nicht im Haus.

In dringenden Fällen können Sie sich natürlich gerne an meine Kollegen wenden.
Sie erreichen uns unter folgender E-Mail-Adresse: sup...@syndesk-sw.de.

Außerdem steht Ihnen  Johannes Obermann unter der E-Mail johannes...@syndesk-sw.de zur Verfügung.

Wir melden uns bei Ihnen schnellstmöglich zurück.

Mit freundlichen Grüßen

Martin Köditz
E-Mail: martin....@syndesk-sw.de Festnetz: +49 5131 46358-300 Mobil: +49 174 9095174	Adresse: Dieselstraße 18 30827 Garbsen	Geschäftsführer: Jörg Obermann, Martin Köditz USt.-ID: DE357564599 Gerichtsstand: Amtsgericht Hannover HRB 224176
Vertraulichkeitshinweis: Die in dieser E-Mail enthaltenen Informationen sind vertraulich zu behandeln und sind nur für die Personen oder das Unternehmen bestimmt, an welche sie tatsächlich gerichtet sind. Sollten Sie diese Nachricht aufgrund eines Übermittlungsfehlers erhalten haben, bitten wir Sie den Versender der Nachricht unverzüglich zu informieren. Ebenso bitten wir Sie, den Inhalt Dritten gegenüber vertraulich zu behandeln und ihn nicht weiter zu verbreiten.   Sicherheitshinweis: Das Internet ist kein sicheres Kommunikationsmedium. Im Rahmen unseres Qualitätsmanagements und aller gebotenen Sorgfalt wurden Schritte eingeleitet die einen Virenbefall dieser E-Mail weitgehend ausschließen, aber wegen der Beschaffenheit des Übertragungsmediums nicht garantiert werden können.

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-suppo...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/firebird-support/66a5b2d3-20d0-4c40-8030-69a0870ce57c%40ibphoenix.com.
⚠️ ACHTUNG!

Die Nachricht kommt von einem externen Absender. Seien Sie achtsam beim Öffnen von Links und Dateianhängen.

[Tomasz Dubiel]

select A.MON$SERVER_PID, A.MON$REMOTE_PID, M.MON$MEMORY_USED / 1024 / 1024 as USED_MB,
       M.MON$MEMORY_ALLOCATED / 1024 / 1024 as ALLOCATED_MB
from MON$ATTACHMENTS A
join MON$MEMORY_USAGE M on A.MON$STAT_ID = M.MON$STAT_ID
where A.MON$ATTACHMENT_ID <> current_connection
order by M.MON$MEMORY_USED desc  

[Mark Rotteveel]

Is that server-side or client-side? If server-side, maybe the statement
cache? If client-side, maybe long running transactions and ineffective
use of the inline blob cache which leads to blobs not getting evicted?
(Though I would expect that memory to be reclaimed once the connection
is closed, both for statement cache and inline blob cache).

Do you have any custom settings in firebird.conf, what are they?

Otherwise, there might be a memory-leak somewhere.

--
Mark Rotteveel

[Köditz, Martin]

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-suppo...@googlegroups.com.

To view this discussion, visit https://groups.google.com/d/msgid/firebird-support/467d4c59-842c-43be-a4d4-93fb3e1d163c%40lawinegevaar.nl.

[Tomasz Dubiel]

There are no long running transactions. firebird.conf taken from ib-aid calculator.

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 11.02.2026 12:44:
> select A.MON$SERVER_PID, A.MON$REMOTE_PID, M.MON$MEMORY_USED / 1024 / 1024 as
> USED_MB,
>        M.MON$MEMORY_ALLOCATED / 1024 / 1024 as ALLOCATED_MB
> from MON$ATTACHMENTS A
> join MON$MEMORY_USAGE M on A.MON$STAT_ID = M.MON$STAT_ID
> where A.MON$ATTACHMENT_ID <> current_connection
> order by M.MON$MEMORY_USED desc

So, now you have attachments that allocated 5 GB of memory. Look deeper on
details to see what exactly memory is used for, as Mark said.

--
WBR, SD.

[Mark Rotteveel]

On 11/02/2026 12:47, Tomasz Dubiel wrote:
> There are no long running transactions. firebird.conf taken from ib-aid
> calculator.

And those values are?
--
Mark Rotteveel

[Tomasz Dubiel]

#Configuration for Firebird 5 (vanilla) Classic (64 bit)
# 8 vCPU, 40GB ram, 5GB database, pagesize 16384, 60 users

ServerMode = Classic # Firebird Classic
DefaultDBCachePages =2048
LockHashSlots = 65519  # slots
LockMemSize  = 30M
ParallelWorkers = 1 # default parallel threads
MaxParallelWorkers = 64 # parallel threads for sweep, backup, restore
MaxStatementCacheSize=32M
OuterJoinConversion = true
OptimizeForFirstRows = false
UseFileSystemCache = true
TempCacheLimit = 86M
RemoteServicePort = 3050
InlineSortThreshold = 16384 # use REFETCH plan for big sortings

ExtConnPoolSize = 64 # external connections pool size
ExtConnPoolLifeTime = 3600 # seconds

#set DataTypeCompatibility according Migration Guide https://ib-aid.com/download/docs/fb5migrationguide.html
#DataTypeCompatibility =
#WireCryptPlugin = ChaCha64, ChaCha, Arc4
WireCrypt = Enabled
#WireCompression = false
#RemoteAuxPort = 0
#authentication plugin setup
#Recommendation - use SELECT * FROM SEC$USERS
#to check that you have users for all plugins
AuthServer = Srp256, Legacy_Auth
UserManager = Srp, Legacy_UserManager

#MaxIdentifierByteLength = 252
#MaxIdentifierCharLength = 63
#DefaultTimeZone =
#SnapshotsMemSize = 64K # bytes
#TipCacheBlockSize = 4M # bytes
ReadConsistency=0
DataTypeCompatibility = 3.0
OuterJoinConversion = false

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 11.02.2026 12:50:
> # 8 vCPU, 40GB ram, 5GB database, pagesize 16384, 60 users

Well, you requested config for Firebird taking 40GB of RAM and it is taking
40GB of RAM. Exactly what you wanted.

--
WBR, SD.

[Alexey Kovyazin]

Hello Tomasz,

Blob operations can inflate memory. 

Regards, 

Alexey Kovyazin 

Dimitry Sibiryakov' via firebird-support <firebird...@googlegroups.com>:

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-suppo...@googlegroups.com.

To view this discussion, visit https://groups.google.com/d/msgid/firebird-support/247f2138-2a2e-4ada-856c-bac1cf06a12f%40ibphoenix.com.

[Dimitry Sibiryakov]

Alexey Kovyazin wrote 11.02.2026 13:05:
> Blob operations can inflate memory.

If they do - this is a bug in application.

--
WBR, SD.

[Mark Rotteveel]

Thanks, nothing jumps out, though I do think the MaxParallelWorkers=64
might be a bit much for Classic (it's per server process!), but given
ParallelWorkers = 1, that shouldn't matter (unless you override it at
the connection level or in databases.conf).

InlineSortThreshold also seems a bit high.

In any case, say 40GB, with 100 connections, then that is 0.4GB per
connection, lets say 400MB.

Based on these settings alone you have:

page cache: 2048 * 16384 = 32 MB
statement cache: 32 MB
temp cache limit: 86 MB
---
total: 150 MB per connection for those caches alone

And I'm not sure if that temp cache limit is a hard limit per
connection, or on some other lower level (which could mean it consumes
multiples of that 86MB?), similarly I imagine that the statement cache
might have some additional overhead not counted towards that 32 MB.

Then there is the TIP cache, which takes 4 MB per block, and the amount
of blocks depends on the transaction gap. One block of 4 MB fits
512*1024 transactions, though depending on the transaction numbers, it
might straddle two blocks even if it's less than that.

Then there is memory for the external connections cache (if you use
external connections), and possibly their inline blob caches.

And then you also have all the memory used during normal operation,
blobs, pending rows, prepared statements, etc.

So, reaching 40 GB itself doesn't seem far fetched (especially not if
you sized the configuration on using 40 GB). However, the fact that some
connections take 1 to 4 GB (or even higher) seems a bit high.

You may want to drill down in the information of MON$MEMORY_USAGE to see
where it's spent (as right now, you're only looking at the overall
memory usage of a connection).

Mark

> --
> Support the ongoing development of Firebird! Consider donating to the
> Firebird Foundation and help ensure its future. Every contribution makes
> a difference. Learn more and donate here:

> https://www.firebirdsql.org/donate <https://www.firebirdsql.org/donate>

> ---
> You received this message because you are subscribed to the Google
> Groups "firebird-support" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to firebird-suppo...@googlegroups.com

> <mailto:firebird-suppo...@googlegroups.com>.

> To view this discussion, visit https://groups.google.com/d/msgid/
> firebird-

> support/019eb309-5e84-4ef0-9fcd-577d5bed5247n%40googlegroups.com
> <https://groups.google.com/d/msgid/firebird-
> support/019eb309-5e84-4ef0-9fcd-577d5bed5247n%40googlegroups.com?
> utm_medium=email&utm_source=footer>.


--
Mark Rotteveel

[Tomasz Dubiel]

Are you talking about Firebird?

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 11.02.2026 13:33:
> Are you talking about Firebird?

"Application" in this context is a client program that works with a database.
I.e. - your program.

--
WBR, SD.

[Tomasz Dubiel]

There is 63 GB of RAM on this server. And Firebird uses more and more RAM. It ends with OOM killer on Linux.

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 11.02.2026 13:41:
> There is 63 GB of RAM on this server. And Firebird uses more and more RAM. It
> ends with OOM killer on Linux.

Right. That's why, as Mark said, you must dig deeper, find the problem and
fix it now.

--
WBR, SD.

[Tomasz Dubiel]

What exactly would be improper usage of blobs? And what should be changed?

Best regards.

[Tomasz Dubiel]

I remind the database is only 1 GB.

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 11.02.2026 13:48:
> What exactly would be improper usage of blobs? And what should be changed?

This is not a question you must care now. Now you must find out what exactly
memory is used for. Analyze data from MON$MEMORY_USAGE at all levels/objects for
especially hungry attachments.

--
WBR, SD.

[Tomasz Dubiel]

We know what processess consume all the time more and more data. They are part of REST service, they do many things. How can we find out what mechanism is precisely responsible for RAM growth?

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 11.02.2026 14:05:
> We know what processess consume all the time more and more data. They are part
> of REST service, they do many things. How can we find out what mechanism is
> precisely responsible for RAM growth?

Table MON$MEMORY_USAGE contain usage of memory for transactions, statements
and requests (calls). Select data for them to find out which exactly
statement(s) and/or transaction eats the RAM.
Then analyze why your REST service doesn't free these resources.

--
WBR, SD.

[Carlos H. Cantu]

In Firebird, BLOBs are immutable. Every time you modify a BLOB inside a Procedure, Trigger, PSQL, etc the engine doesn't "edit" the content—it creates a brand-new copy!

If you update a 1 MB BLOB field three times inside a procedure, etc:

- Firebird generates 3 separate temporary BLOBs.

- Your database file can grow by 3 MB, even if you only changed a single character in the blob.

So, large updates inside loops can cause massive, rapid growth of your .fdb file.

PS: 

- This "wasted" space can/will be reused after the transaction COMMITs.

- I'm being simplistic in the example :)

 

[]s

Carlos

www.firebirdnews.org - www.FireBase.com.br

[Tommi Prami]

One thing comes into mind is that there are non closed connections dangling.

Just a guess.

Would explain problems if using CLassic server, and easy to see if there are way more processes than should have active connections.

-tee-

--

Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate

---
You received this message because you are subscribed to the Google Groups "firebird-support" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebird-suppo...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/firebird-support/75e7c3f8-b06c-4c86-a506-c421f72e809fn%40googlegroups.com.

[Tomasz Dubiel]

I changed query to this:

SELECT ....

FROM mon$transactions s
join MON$ATTACHMENTS A on (s.mon$attachment_id = a.mon$attachment_id)
join mon$memory_usage m on s.mon$stat_id = m.mon$stat_id
join MON$IO_STATS St on (s.MON$STAT_ID = St.MON$STAT_ID)
left join MON$STATEMENTS MST on (s.MON$TRANSACTION_ID = MST.MON$TRANSACTION_ID)
order by M.MON$MEMORY_USED desc;
and it showed that those usages are from idle, read-only transactions.

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 11.02.2026 14:41:
> and it showed that those usages are from idle, read-only transactions.

Now you have a staring point. Go to your REST service and find out what was
done in these transactions and why these transactions weren't finished as soon
as all requested data had been fetched.

--
WBR, SD.

[Dimitry Sibiryakov]

Carlos H. Cantu wrote 11.02.2026 14:36:
> In Firebird, BLOBs are immutable. Every time you modify a BLOB inside a
> Procedure, Trigger, PSQL, etc the engine doesn't "edit" the content—it creates a
> brand-new copy!

And not only this. Transliteration of a text BLOB from a storage charset into
a connection charset does it too.
This is one of main reasons why the old Borland Delphi pattern to have one
global long read-only transaction for shared usage was totally bad.

--
WBR, SD.

[Tomasz Dubiel]

Did something change between FB 3.0 and FB 5.0? I ask, because such read-only transactions (idle) were being created and not closed and it didn't cause such problems so far on FB 3.0. There was no memory growth.

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 11.02.2026 16:03:
> Did something change between FB 3.0 and FB 5.0? I ask, because such read-only
> transactions (idle) were being created and not closed and it didn't cause such
> problems so far on FB 3.0. There was no memory growth.

Yes: read consistency mode for read committed transactions was introduced.

--
WBR, SD.

[Tomasz Dubiel]

Thats not the case for us. We disable this. 

--
Support the ongoing development of Firebird! Consider donating to the Firebird Foundation and help ensure its future. Every contribution makes a difference. Learn more and donate here:
https://www.firebirdsql.org/donate
---
You received this message because you are subscribed to the Google Groups "firebird-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-suppo...@googlegroups.com.

To view this discussion, visit https://groups.google.com/d/msgid/firebird-support/17d7caaf-dbd4-4d72-bf3b-3749072fc808%40ibphoenix.com.

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 11.02.2026 16:16:
> Thats not the case for us. We disable this.

Then you have no choice but really to look for the problem instead of looking
for random workarounds.

--
WBR, SD.