Importing excel file with Arabic Text into MapInfo

[akammer]

Dear all,

I have been struggling with Arabic text for some time now.

I am an English speaker living in the Middle East and our data is
mixed - sometimes Arabic and sometimes English or both in the
Attribute data.

We use the English versions of Windows and Mapinfo as well as ArcView.

If I use the translation wizard tool and translate shape files to
Mapinfo Tables, I can see strange looking text in my attribute data
where Arabic text should be displayed. If I then press f8 and display
my table in Arabic Transparent text, I can see the real Arabic text
just fine.

BUT... how on earth can I import data from an excel file into Mapinfo
without completely screwing up my Arabic text? I have tried opening
the excel file directly and converting it to a CSV or text file. I
have tried ANSI and Unicode as import text, but my text is always
displayed as ?????? - question marks.

PLEASE can anyone help me?

[Warren Vick]

Hi,

At the moment, MapInfo TAB files don't have the ability to store Unicode, so old-school double-byte encoding is used with a variety of character sets. The problem there is that you can never mix, say, Arabic and Japanese together. To make an English Windows+MapInfo Pro combination work with Arabic data, you need to do a little trickery.

1) Install Windows language/region pack(s) for Arabic.
2) Download AppLoc from Microsoft - a aging tool now, but it still works. I think you need to be an Administrator to install it.
3) Run AppLoc... chose MapInfoW.exe as Pro's executable and Arabic locale. This will fool MapInfo Pro into thinking it's running on a Arabic version of Windows.

We have lots of data in Arabic stored in TAB files, but I've never brought anything in to Access. I think it should work though.

The tip of using AppLoc came from Eric Blasenheim so I tip my hat to him every time I pass it on!

It's perhaps never going to be PB's highest priority, but for me, support for Unicode would be a great addition to TAB/Pro.

Regards,
Warren Vick
Europa Technologies Ltd.
http://www.europa.uk.com

--
You received this message because you are subscribed to the Google Groups "MapInfo-L" group.To post a message to this group, send email to mapi...@googlegroups.com To unsubscribe from this group, go to:
http://groups.google.com/group/mapinfo-l/subscribe?hl=en
For more options, information and links to MapInfo resources (searching archives, feature requests, to visit our Wiki, visit the Welcome page at http://groups.google.com/group/mapinfo-l?hl=en

[Manoj Reghupathy]

Try this:

Use commit statement.

Commit table newtab as "  "Type Native Charset "WindowsArabic"

 in Mapbasic window while saving .

--

Regards

___________________________________________________

 

Manoj Reghupathy

 

 

[Eric Blasenheim]

These issues can be a bit tricky to understand so I would need to clarify a few points.  While Warren's points are correct, if the right system settings are in place, sharing Arabic and English in the same file is not a problem.  That is, as long as the English is just the ASCII set of characters, all can be fine.

 

1) Are you running Windows with Arabic set as your locale, language?  In particular, the setting for non-Unicode programs is relevant to behavior in Professional. If you are, then the AppLoc solution mentioned by Warren is not relevant. If your setting is for Western Latin, often called Latin1, then you will either need to change your setting or use AppLoc. The language version of Professional is not important. That only handles the language of the user interface. 

 

2) It is important to recognize the difference in whether the data is correct or not versus whether there is a display, usually a font issue. You mentioned that changing the font made things work but then you said you needed a solution that did not screw up Arabic text. So I am bit confused by those two statements. The first thing to check is whether the .TAB file indicates Windows Arabic. There should be two charset clauses in the .tab. One is for the .tab file text itself (like the metadata) and the other for the actual data.  If these do not say Windows Arabic then no font solution is going to fix things. 

 

3) These MapInfo tables were created by something you called the "translation wizard". Is this the Universal Translator? .I am also not familiar with how .SHP files handle language encoding so I can't be sure that the data was transferred correctly.  I do know that some .SHP files can be stored as UTF-8 although I don't know that it says this anywhere.   SHP files can also be opened directly inside Professional and there is a UTF-8 choice there. I believe the Universal translator would expect that the source data is in the same encoding. That is, the Universal Translator does not do character encoding translation whereas Professional will do that encoding transformation.

 

3) If the data is correct but the display is wrong, then the solution should be much easier; just ensuring that the fonts support Windows Arabic characters. When you see a "?" character that means that the substitution was done by Windows. If you see an undescore "_" character the substitution was done by Professional because a character conversion failed.

 

Let us know the answers and perhaps we can help. Please feel free to contact support, of course.

 

Eric Blasenheim

Pitney Bowes Software

[Warren Vick]

As always, a comprehensive response from Eric! Thanks.

 

FYI, Shapefiles use a .CPG (code page) file to declare the character encoding used. It's just a one-line text file. e.g. "UTF-8".

 

Regards,

Warren Vick

Europa Technologies Ltd.

http://www.europa.uk.com

 

--

[Uffe Kousgaard]

For UTF-8 that would be the text "65001", not "UTF-8".

 

Regards

Uffe Kousgaard

[Eric Blasenheim]

I was not aware of .CPG files. I have not come across them before.  I guess most files that I have seen assume you don't need them. Thanks for the info. By the way, 65001 is the Microsoft API id for UTF-8. 

 

Eric Blasenheim

Pitney Bowes Software 

 

[Warren Vick]

Hi Uffe,

 

Well, I don't know about "65001" but all our CPG files (written by FME) say "UTF-8" and work just fine!

 

Regards,

Waren Vick

Europa Technologies Ltd.

http://www.europa.uk.com

 

--

[Annelene Kammer]

Thanks so much for everyone's help! It seems to work now that I have
followed all the steps.

Kind regards

Annelene Kammer-Tischendorf

Sent from my iPad

[Ahmet DABANLI]

Dear All,
I've worked with Arabic text in Saudi, and Turkish in Turkey,
For some projects we have to work in three language,
For Example, I need a file to be stored in "WindowsTurkish" charset,
and another in "WindowsArabic"
I do it manually for one of them because the non unicode can be set
either to Arabic or Turkish.

If I make manual settings, then open the files, sometime it is not a problem,

but if by mistake, i save an arabic file while i'm in turkish locale,
then the chars are lost.
For wrong display issues, we were changing the regedit-
hklm-software-microsoft-windows- current version- font substitudes,
and set Arial,0 = Arial,172
so default display comes automatically to Arabic, or Turkish
I wonder is it possible to have a solution or workaroud for this, so i
can work both in Arabic and Turkihs in the same time for different
tables,

Ahmet DABANLI

[Eric Blasenheim]

Ahmet,

 

I cannot think of a way to easily do this. MapInfo Professional works with one "charset" at a time. In this context, "charset" is equivalent to one Windows codepage and Arabic and Turkish are separate codepages. As Turkish is a Latin based codepage (Latin5) all of the base ASCII characters will work fine in an Arabic environment but all the special characters of Turkish would not be available in the Arabic environment.

 

Eric Blasenheim

Pitney Bowes Software

On Wednesday, May 9, 2012 5:28:32 AM UTC-4, akammer wrote:

[Ahmet DABANLI]

Dear Eric, I dont mean having two charset in one table,
Let me try to explain it in more detail.

Let say I have two tables, with different charsets, one is arabic, and
one is turkish.
If the regional and non unicode setting in Windows is set for Turkish
. If I open Arabic Table andmake changes and save, I lost arabic
chars.
If i change the settings to arabic, and open Turkish files, and make
changes and save, then i lost turkihs chars.
When i want to change font style of the browser then i can not see
turkihs chars.

Or,
If i save a query of arabic charset table, as a new file, the charset
set to windows turkihs or neutral, but it should stay as arabic.
best regards

[Eric Blasenheim]

When I said "one charset at a time", I meant one session of Professional and was not thinking of one table.  In a non-Unicode setting, the Turkish and Arabic specific characters occupy the same memory space (the values > 127 and < 256) and therefore cannot be distinguished from each other.  That is why the language setting allows programs to assume one or the other but not both at the same time.

Eric Blasenheim

Pitney Bowes Software