Problem with Polish characters

[Tomasz Dubiel]

Hello.

What can be the problem with displaying Polish characters from the database? There is Windows server with Firebird 3 databases. When I run there any program to read the data from some tables, I get only something like on the screenshot:

It looks like it's not a problem with data, because if I connect to this database from other computers, everything is ok.

Server was originally installed with English language, but country, region and language is already changed to Polish, however data from the database at this server are read like on the screenshot.

Best regards,

Tomek.

[Tomasz Tyrakowski]

Possible solution (worked for me so far) is to set the proper locale for
non unicode-aware applications. You need to get to advanced
(administrative) language settings, and set the locale for non-unicode
apps to Polish (see the attached screenshot).
It happens when Windows had been installed in another language and
localized later.

regards
Tomasz

[Tomasz Dubiel]

I have set this to Polish.

[Tomasz Tyrakowski]

In that case make sure the font used in your application (on that very
machine) is actually capable of displaying Polish diacritics.
In my experience, setting the locale for non-unicode apps has always
solved this kind of problem, but maybe the machine has other faults
(e.g. the locale change wasn't completed for some reason and not all
system assets have been actually localized).
The problem may be in TTF/OTF files themselves, or in registry settings
for fonts and font substitutes (maybe compare
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows
NT\CurrentVersion\FontSubstitutes on the faulty machine with the
equivalent registry key on a working machine).

regards
Tomasz

[Mark Rotteveel]

What is the character set of the column, and what is the connection
character set?

Mark
--
Mark Rotteveel

[Tomasz Dubiel]

WIN1250. This is something with the server, because as I said, data read from other computers are presented correctly.

I will check Tomasz's advice.

Best regards.

[Tomasz Tyrakowski]

On 09.08.2024 at 10:37, Tomasz Dubiel wrote:
> WIN1250. This is something with the server, because as I said, data read
> from other computers are presented correctly.
> I will check Tomasz's advice.

To be exact, you need to check the faulty _client_ machine (one that
displays incorrect characters in the application). Server has nothing to
do with it.
And just in case, like Mark suggested, make sure the connection charset
(on the client) is indeed WIN1250.

regards
Tomasz

[Tomasz Dubiel]

That's the case - the data are presented wrong on the database server (IBExpert, our ERP system). When we run our ERP system on clients, the data are presented correctly.

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 09.08.2024 12:01:
> That's the case - the data are presented wrong on the database server (IBExpert,
> our ERP system). When we run our ERP system on clients, the data are presented
> correctly.

It means that your ERP system is not protected from humen mistakes and
someone inserted data in codepage 850 (or some other) to database where 1250 is
expected.
Check byte representation of the data in fields by casting it into OCTETS.

--
WBR, SD.

[Tomasz Dubiel]

That would be weird. It's the rule for every field in the database - even those which are predefined.

Can you provide an example of casting to octets?

Best regards.

[Tomasz Dubiel]

The char in this case is "ę"

[Tomasz Tyrakowski]

On 09.08.2024 at 12:01, Tomasz Dubiel wrote:
> That's the case - the data are presented wrong on the database server
> (IBExpert, our ERP system). When we run our ERP system on clients, the data
> are presented correctly.

That's still the client machine and client software that needs to be
checked. Which, in this particular scenario, also happens to be the
machine running the database server :)
But let's not make an issue of it, I get what you mean, I must have
missed the part that the application displaying garbage runs on the same
machine as FB server.
Are you sure _the same_ data (i.e. the same rows from the same tables)
is displayed correctly on other Windows machines? Because if not, then
it might be a problem with data after all (and not with Windows setup).

regards
Tomasz

[Tomasz Dubiel]

I'm sure. This happens for even basic fields as the default place of sale, of buy, default warehouse (Główne), the name of the customer (dane firmy), etc..

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 09.08.2024 12:11:
> Can you provide an example of casting to octets?

Example: cast(f as varbinary(100)).

--
WBR, SD.

[Tomasz Dubiel]

Are you sure for varbinary? Not recognised syntax.

[Tomasz Dubiel]

It's probably available from Firebird 4. This is Firebird 3.

https://github.com/FirebirdSQL/firebird/issues/5351

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 09.08.2024 12:24:
> Are you sure for varbinary? Not recognised syntax.

What is your Firebird version? On old ones it is "varchar(100) character set
octets".

--
WBR, SD.

[Tomasz Dubiel]

Byte representation is identical. I check on the other database, but I compare the same predefined value.

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 09.08.2024 12:32:
> Byte representation is identical. I check on the other database, but I compare
> the same predefined value.

Then the problem is not on server but on client side.

--
WBR, SD.

[Tomasz Tyrakowski]

On 09.08.2024 at 12:21, Tomasz Dubiel wrote:
> I'm sure. This happens for even basic fields as the default place of sale,
> of buy, default warehouse (Główne), the name of the customer (dane firmy),
> etc..

OK, so if other machines with the same client software show Polish
diacriticts correctly in the same data, and only on this single Windows
system the same apps show garbage, then it's either invalid client
charset (but you've probably already double-checked that), or faulty
Polish localization of Windows.
I don't think there's anything wrong with the data in the database. I'd
focus on checking the fonts and locale settings on this machine.
Maybe also try to set the locale for non-unicode applications to English
and then back to Polish (it's Windows, so you never know ;) ).
IIRC each such change requires a reboot, but if possible, I'd give it a try.

regards
Tomasz

[Tomasz Dubiel]

Yeah, it all seems to be Windows problem. I will do it. I will try at first with simple Windows restart. If it doesn't help, I will try with resetting this locale for non-unicode application.

Best regards.

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 09.08.2024 12:36:
> Yeah, it all seems to be Windows problem.

Actually the problem is your ERP application. I guess it is written on
ancient Delphi version like 7 or 2006.

--
WBR, SD.

[i...@kaabnl.nl]

Try opening the database (on server) with something like IBO, and check those values.

I tend to agree that it's on client side (i work in PL with Delphi/Firebird). In older apps i have the same.

M.

--
You received this message because you are subscribed to the Google Groups "firebird-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebird-suppo...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/firebird-support/4aa41476-673b-4617-8730-e4c5632e4f4dn%40googlegroups.com.

Virus-free.www.avg.com

[Tomasz Dubiel]

It is at least Delphi 10.2.3

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 09.08.2024 12:40:
> It is at least Delphi 10.2.3

In this case it uses Unicode for display and settings for non-Unicode
programs must be irrelevant. Something is wrong inside of the application.

--
WBR, SD.

[Tomasz Dubiel]

In IBExpert as well? I already said the sane occurs in IBExpert also.

[Dimitry Sibiryakov]

Tomasz Dubiel wrote 09.08.2024 12:43:
> In IBExpert as well?

IBExpert is written on Delphi 7. It is non-Unicode program.

--
WBR, SD.

[Tomasz Dubiel]

We already had such cases when locale for non unicode apps was English and changing it to Polish fixed the problem. I would bet this to be Windows problem after all :-)

[Dimitry Sibiryakov]

Once again: locale for non-Unicode programs must not affect application built
with Delphi 2009+ because they ARE Unicode programs.

--
WBR, SD.

[Tomasz Tyrakowski]

Not necessarily. You can still use AnsiStrings when communicating with
external libraries / units that require single byte encoding, and you
can set the encoding of the single-byte strings in project settings (in
code generation / code page). Then all explicit casts of Delphi "string"
variables to "AnsiString" type will be translated to the chosen code page.
So, tehcnically speaking, while all "string" variables in modern Delphi
are represented internally as unicode, the single-byte strings are still
available via casting to AnsiString.

regards
Tomasz

[Ertan Küçükoglu]

Tomasz Dubiel <tkdu...@gmail.com>, 9 Ağu 2024 Cum, 13:40 tarihinde şunu yazdı:

It is at least Delphi 10.2.3

I have no idea how difficult it would be for you, but I would think about converting everything to UTF8. 

[Mark Rotteveel]

On 09/08/2024 10:37, Tomasz Dubiel wrote:
> WIN1250. This is something with the server, because as I said, data read
> from other computers are presented correctly.

What happens if you specify UTF8 as the connection character set? The
fact it shows the Unicode replacement character to me suggests the
application took the contents of the column as bytes, and then tried to
decode it as UTF-8.

Mark
--
Mark Rotteveel

[Tomasz Dubiel]

With UTF-8 everything is ok.

[Tomasz Dubiel]

This is checked in IBExpert. Our application always uses WIN1250.

[Mark Rotteveel]

On 09/08/2024 14:35, Tomasz Dubiel wrote:
> With UTF-8 everything is ok.

Have you checked which version of fbclient actually gets loaded? Maybe
that is doing something weird?

Mark
--
Mark Rotteveel

[Tomasz Dubiel]

ERP application uses everywhere the same fbclient. When it comes to IBExpert, it uses some other one, but the effect is the same.

[Tomasz Dubiel]

Unfortunately, after setting everything to Polish/Poland and restarting the server, the problem persists.

[Tomasz Dubiel]

Update:

The problem went away when unchecked: "Beta: Use Unicode UTF-8 for worldwide language support".

Best regards,

Tomek.