Unable to allocate memory from operating system

[Ertan Küçükoglu]

Hello,

I am using Firebird 4.0.2 and 4.0.3 32bit windows versions. Firebird server installations are custom (release setups are not used).

There are several solutions and all of them are using the same custom FirebirdSQL installation. All of the solutions have windows service and GUI parts and their own separate databases. All connections are over TCP/IP to the localhost using a custom port. Database sizes are very small. Less than 100MB on average. I have seen this error on Windows 10 (Version 21H1, OS Build 19043.1706, 64-bit Edition), 8GB RAM and I don't have access to all problematic systems. There is another system that I saw the error is Windows 10 64bit 22H2, 8GB RAM .

If any given computer runs one single solution (let's say einvoice solution including its specific windows service, GUI and its specific database) all is fine.

If we install another different solution (let's say ocr solution including its specific windows service, GUI and its specific database) on the same computer there are random errors. Not always, but frequently.

All solutions try to install FirebirdSQL on their own directories like C:\solution1 or C:\solution2. Database files are located like C:\solution1\db\db1.fdb C:\solution2\db\db2.fdb

The first installed solution sets itself as a service. Second installation does not add another FirebirdSQL server.

So far I could extract below two from log files:

- unable to allocate memory from operating system

- unable to allocate memory from operating system IAttachment::openCursor failed when loading mapping cache

I was looking for an error in Devart UniDAC components, but they work just fine on a single solution for months. They have not been updated recently. Error occurs at the time of database connection establishment. There is no large data reading SQL or something. This is a Delphi application and one of the errors is more like C++ error.

Errors occur if more than one solution is installed. Errors displayed in a message dialog using GUI applications. Windows services have random triggers. These services are also connecting to the same database and I see the same errors in their logs. firebird.exe active RAM usage reported as 32MB (values from regular task manager) at the time of error.

Below are FirebirdSQL server configuration files

firebird.conf

ServerMode = Super
DefaultDbCachePages = 50K # pages (SuperServer) - increase pages in databases.conf, not heretabases.conf, not here

#AuditTraceConfigFile =

LockMemSize = 20M # bytes (SuperServer)
LockHashSlots = 40099 # slots

MaxUnflushedWrites = -1 # default for posix (non-Windows)
MaxUnflushedWriteTime = -1 # default for posix (non-Windows)
UseFileSystemCache = true
TempCacheLimit = 128M
RemoteServicePort = 13050
InlineSortThreshold = 16384 # use REFETCH plan for big sortings

ExtConnPoolSize = 64 # external connections pool size
ExtConnPoolLifeTime = 3600 # seconds

#set DataTypeCompatibility according Migration Guide https://ib-aid.com/download /docs/fb4migrationguide.htmlm/download /docs/fb4migrationguide.html
DataTypeCompatibility = 3.0
#WireCryptPlugin = ChaCha64, ChaCha, Arc4
#WireCrypt = Enabled (for client) / Required (for server)
#WireCompression = false
#RemoteAuxPort = 0
#authentication plugin setup
#Recommendation - use SELECT * FROM SEC$USERS
#to check that you have users for all plugins
AuthServer = Srp256, Legacy_Auth
UserManager = Srp, Legacy_UserManager

#MaxIdentifierByteLength = 252
#MaxIdentifierCharLength = 63
#DefaultTimeZone =
#SnapshotsMemSize = 64K # bytes
#TipCacheBlockSize = 4M # bytes

databases.conf

# ------------------------------
# List of known databases
# ------------------------------
#
# Makes it possible to specify per-database configuration parameters.
# See the list of them and description on file firebird.conf.
# To place that parameters in this file add them in curly braces
# after "alias = /path/to/database.fdb" line. Example:
# big = /databases/bigdb.fdb
# {
# LockMemSize = 32M # We know that bigdb needs a lot of locks
# LockHashSlots = 19927 # and big enough hash table for them
# }
#

#
# Example Database:
#
employee.fdb = $(dir_sampleDb)/employee.fdb
employee = $(dir_sampleDb)/employee.fdb

#
# Master security database specific setup.
# Do not remove it until you understand well what are you doing!
#
security.db = $(dir_secDb)/security4.fdb
{
RemoteAccess = false
DefaultDbCachePages = 256
}

#
# Live Databases:
#

FirebirdSQL server is run with below parameters in windows services

c:\solution1\firebird.exe -i -s CustomServiceName

Directory to firebird.exe changes according to which solution was installed first. 

What I observed so far is that two different databases cannot be connected at the same time. Any single one of them works OK on computers having errors. In order to find out this I had to manually turn off windows service applications to be sure that just one database has an active connection.

If I setup a virtual system Windows 10 64bit 22H2, 8GB RAM and test on it using production setup files. I didn't get any errors with more than one solution. Any single one of them or both can run just fine. Either of them can be run first. I could not figure out what might be the problem or where to check for clues.

Any help is appreciated.

Thanks & Regards,

Ertan

[Dimitry Sibiryakov]

Ertan Küçükoglu wrote 13.10.2023 23:53:
> DefaultDbCachePages = 50K # pages (SuperServer) - increase pages in
> databases.conf, not heretabases.conf, not here

With page size 16 this setting will use 0,8Gb memory which is too big for 32
bits process. Use 64 bits Firebird.

--
WBR, SD.

[Mark Rotteveel]

32-bits processes on Windows can allocate 2GiB (or 4GiB when they are
large-address-aware, though Firebird isn't AFAIK), so why is 800MiB too
large?

Mark
--
Mark Rotteveel

[Dimitry Sibiryakov]

'Mark Rotteveel' via firebird-support wrote 14.10.2023 9:39:
>>> DefaultDbCachePages = 50K # pages (SuperServer) - increase pages in
>>> databases.conf, not heretabases.conf, not here
>>
>>    With page size 16 this setting will use 0,8Gb memory which is too big for
>> 32 bits process. Use 64 bits Firebird.
>
> 32-bits processes on Windows can allocate 2GiB (or 4GiB when they are
> large-address-aware, though Firebird isn't AFAIK), so why is 800MiB too large?

First, page cache is not the only thing that needs memory.
Second, it is per database so second opened database will double allocation.

That's why I always say that configuration tuning must be the result of the
real system monitoring, not some abstract calculations.

--
WBR, SD.

[Mark Rotteveel]

On 14-10-2023 10:47, 'Dimitry Sibiryakov' via firebird-support wrote:
>   First, page cache is not the only thing that needs memory.
>   Second, it is per database so second opened database will double
> allocation.
>
>   That's why I always say that configuration tuning must be the result
> of the real system monitoring, not some abstract calculations.

Given the description, it seems the OP is using a Firebird server per
database, and memory allocation is per-process.

Mark
--
Mark Rotteveel