Arabic localization Achieved on android

[GodOfDeath]

Hi All,

For last one month I've been trying hard to get this arabic
localization to work, by Arabic Localization I mean complete support
for the language, now my 1.5 cupcake code has following things

1. When I change the phone language to Arabic-Egypt from settings
application it displays localized string for Arabic language.(of
course one has to add values-ar folder with strings.xml in ever app
for this).

2. RTL support for Arabic strings for Label,TextView etc, all arabic
strings will be Identified and automatically it will display it from
right to left.

3. Complete IME for arabic toggles between English also.

4. All the Arabic language strings are Reshaped as per arabic syntax n
semantics.


Regards
God_of_death

Android rocks!!! \w/

[Shachar Shemesh]

Hi,

GodOfDeath wrote:

Hi All,

For last one month I've been trying hard to get this arabic
localization to work, by Arabic Localization I mean complete support
for the language, now my 1.5 cupcake code has following things

1. When I change the phone language to Arabic-Egypt from settings
application it displays localized string for Arabic language.(of
course one has to add values-ar folder with strings.xml in ever app
for this).
  

Did you have to do anything to make the option appear?

2. RTL support for Arabic strings for Label,TextView etc, all arabic
strings will be Identified and automatically it will display it from
right to left.

3. Complete IME for arabic toggles between English also.

  

Which keyboard did you use?

4. All the Arabic language strings are Reshaped as per arabic syntax n
semantics.

  

Can you give an outline of what you had to do? Is there any code changes you can share?

Did you resolve some of the l10n problems? For example, did you get the calender to display the days right to left and not start the week on a Monday?

Thanks,
Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting Ltd.
http://www.lingnu.com

[GodOfDeath]

Hi Shachar,

Did you have to do anything to make the option appear?

Well Itried to find out from android team if RTL was supported but
sadly their ans was no and they stopped after that only, so I did a
little research of my own on the code and came up with a work around
in code.

Which keyboard did you use?

Well I saw one Greek IME available online and modofied it for Arabic
with prediction too.

Can you give an outline of what you had to do? Is there any code
changes
you can share?

Well I got Arabic Reshaper Utilities from Amr Ismail gawish, this guy
has provided it as open source, using this at system level I made
changes and got the said result.

Did you resolve some of the l10n problems? For example, did you get
the
calender to display the days right to left and not start the week on a
Monday?

I've not looked into it, but now that u've brought it to my notice
I'll look into it, but if ur wondering if Android has some sort of
mirroring etc I think the answer is no.

I hope I could be of some help :)

regards
Vikram

[Shachar Shemesh]

GodOfDeath wrote:

Hi Shachar,

Did you have to do anything to make the option appear?
Well Itried to find out from android team if RTL was supported but
sadly their ans was no and they stopped after that only, so I did a
little research of my own on the code and came up with a work around
in code.

Which keyboard did you use?

Well I saw one Greek IME available online and modofied it for Arabic
with prediction too.
  

Try AnySoftKeyboard (http://code.google.com/p/softkeyboard/). The web site does not say it, but the actual keyboard seems to support Arabic (my Arabic is *extremely* rusty, so I didn't actually test this).

Can you give an outline of what you had to do? Is there any code
changes
you can share?

Well I got Arabic Reshaper Utilities from Amr Ismail gawish, this guy
has provided it as open source, using this at system level I made
changes and got the said result.
  

Did you try just writing an app that prints something in Arabic. Once I had the fonts, it worked out of the box for Hebrew for me (which is BiDi but has no shaping). I know it's officially not supported, but it seems to inherit its support from Java, which has fairly decent BiDi support.

I've not looked into it, but now that u've brought it to my notice
I'll look into it, but if ur wondering if Android has some sort of
mirroring etc I think the answer is no.
  

Just ran a test with AnySoft and the Hebrew fonts (I'm not sure whether they were http://firstpost.org/wp-content/uploads/2009/02/hebdroid.zip or something else), but otherwise a vanilla HTC system. These are the results:
Reordering - working great
Editing - working great, including correct left/right arrow (if you don't mind the two cursors when RTL/LTR runs meet, which I, personally, hate)
Mirroring - working great
Shaping - does not seem to work (but, at least my fonts, have Arabic).

It may be, however, that the problem is that my font has the normal Arabic glyphs, but not the shaped ones. The system uses ICU for this task, and I know from experience that ICU does support shaping.

I hope I could be of some help :)
  

I hope I did too.... :-)

[GodOfDeath]

well my friend I'll mail you the screen shots of my work, that would
explain all that is there ;),
sorry m feeling a bit lazy to comment, may be tom I can write every
thing in detail..


regards
Vikram Wanchoo

[TAKEphONE]

Congratulations !

A few questions of my own:

1. Regarding RTL display - did you encounter (or even solve) the
problem with mixed Arabic & numbers - on my Android when I enter
a Hebrew word, then a number - looks like the OS does not detect
the number as a reason to change ordering to LTR, so the number is
displayed reversed: typing 1->2->3 appear 321 (and not 123).
This works ok for mixing Hebrew and English letters -
typing A->B->C appears correctly as ABC.

2. Did you recompile all the source code of 1.5, then flash the device
with the IMG files created ?
Or is there a way to make an UPDATE.ZIP file and use the SD card
update mechanism (for easier deployment) ?

TIA

[GodOfDeath]

Hi Shimon,

1. Well I was facing the problem too but then I used a Reshaper
library in between i.e. I made my widgets use the Reshaper library for
all the text that was being read or edited, this solved the reversing
effect.

2. Well I dont have a device but I did have to compile the whole
source code for my changes since they were at system level, but all
looks good on the Emulator so far.


regards
Vikram Wanchoo

[GodOfDeath]

Hi,

Thanks for your fast response.

1. You mean you can point a place in the source code where we can
enter
a function to fix this issue for all text displays/edits ?

Does this yield a seperate final bin ?
What I mean is - is there as a result of compilation a specific bin
file
you can point to, that if I replace it on a 'real' device - will fix
the
ordering issue ?
(like if I replace the font files on a real device I get them to
show).

2. What version of the source code did you work on ? was it 1.5 ?

************************************************************************************************

Shimon,

1. Yes thats right I've made some changes in the Widgets to get
reshaped Arabic characters, I dont know how exactly you can replace
that on device since I only work on emulator, but should be fairly
simple. I'll think you might have to replace a complete bin/so file
which has all the changes of widgets.

2. I am using 1.5 cupcake version.

hope that helps.

regards
Vikram

[TAKEphONE]

Hi,

Thanks !!!

What I am trying to get at (and correct me if I am talking nonsense)
is -

I already know to get good Hebrew localization on a device, what I
need to
do is:

1. Replace the font files.
2. Replace/add the soft-keyboard with one that supports Hebrew.
3. Fix this numbers ordering issue.

Now - if I create an UPDATE.ZIP file including only these files -
the font files, my keyboard utility, and the BIN file including
changes I make to the widgets source - will a user be able to
save this to an SD card, launch the device in "fastboot" mode,
and update - which will only replace these needed files ?

TIA
Shimon

On Jul 30, 1:51 pm, GodOfDeath <vikram.wanc...@gmail.com> wrote:
> Hi,
>
> Thanks for your fast response.
>
> 1. You mean you can point a place in the source code where we can
> enter
> a function to fix this issue for all text displays/edits ?
>
> Does this yield a seperate final bin ?
> What I mean is - is there as a result of compilation a specific bin
> file
> you can point to, that if I replace it on a 'real' device - will fix
> the
> ordering issue ?
> (like if I replace the font files on a real device I get them to
> show).
>
> 2. What version of the source code did you work on ? was it 1.5 ?
>
> *************************************************************************** *********************
>
> Shimon,
>
> 1. Yes thats right I've made some changes in the Widgets to get

> reshapedArabiccharacters, I dont know how exactly you can replace

[GodOfDeath]

Hi simon,

1. you can change font file in following manner
Rename ur font file to "DroidSansFallback.ttf" and then replace it
with the one in the phone(make sure u take a copy of the orignal).
2. There are many IME(imput method engine)available on the net you can
take one as a reference and develope one for hebrew, its pretty simple
if you have font file in place. If you want more info I can help.

3. Well No issue cant be solved unless u play around with framework
components a little, for me there was no problem with nos since I had
already taken care of RTL at framework level.


regards
Vikram Wanchoo

[TAKEphONE]

Hi,

Already did items 1,2.

I even managed to download an UPDATE.ZIP file with HTC Hero,
and add the Hebrew fonts, plus my own IME keyboard, and update
a device so that it has the fonts and keyboard.

Only thing I am missing is the fix for the numbers ordering.
So I wanted to know if I can fix it in the source, compile the source,
then find the specific BIN files I need to replace in the UPDATE.ZIP.

I cant just use the SYSTEM.IMG file I get from compilation, since
the HTC Magic device I use is not compatible with the Google
open source code (not with the hardware level).

So I thought if I find a ny specific BIN I can replace only it...

BTW - can you share what source files you touched to fix the ordering
(or add the reshaper) ?
What functions ?

Thanks

On Jul 30, 3:28 pm, GodOfDeath <vikram.wanc...@gmail.com> wrote:
> Hi simon,
>
> 1. you can change font file in following manner
>    Rename ur font file to "DroidSansFallback.ttf" and then replace it
> with the one in the phone(make sure u take a copy of the orignal).
> 2. There are many IME(imput method engine)available on the net you can
> take one as a reference and develope one for hebrew, its pretty simple
> if you have font file in place. If you want more info I can help.
>
> 3. Well No issue cant be solved unless u play around with framework
> components a little, for me there was no problem with nos since I had
> already taken care of RTL at framework level.
>
> regards
> Vikram Wanchoo
>
> On Jul 30, 5:09 pm, TAKEphONE <shimo...@gmail.com> wrote:
>
>
>
> > Hi,
>
> > Thanks !!!
>
> > What I am trying to get at (and correct me if I am talking nonsense)
> > is -
>

> > I already know to get good Hebrewlocalizationon a device, what I

[GodOfDeath]

Sure what I did was I modified TextView.java, which is available under
"mydroid\frameworks\base\core\java\android\widget"
you can look at the part where it checks for Gravity property. Also
you might wana add one utility.java file which takes care of your
number i.e iif its a hebrew no or not, on checking u can set the
string being passed to SetText() function,simple.

PS: make sure u check the string is null or not and don't for get to u
use logs they help a lot
hope i could help
regards
Vikram Wanchoo

[G_man]

Salam,
I was really trying to do that for a long time. I will contact you to
get a build with native Arabic as this was my objective. I have two
guys I work with. Amr Gawish the one you used his reshaper and
Abdullah Bahaitham who made 3 Arabic keyboards. I made a build with
Arabic fonts on it and added the keyboard with the build. But had some
problems with reveting. shoot me an email.

Aman Alshurafa

[x2an...@gmail.com]

Hi

Are you planning to release a build, or integrating the Arabic support
to the android project?

i have read somewhere that the android team are not planning to add,
at least soon, fonts that support complex scripts like Arabic, Urdu
etc...

please let us know once you release something

thanks

[Jean-Baptiste Queru]

I don't think that at this point we (Google) are equipped to review,
accept and merge changes to the standard Android fonts.

Improvements in the platform to better handle right-to-left and
bidirectional text and layout are fair game as far as I can guess.

JBQ

--
Jean-Baptiste M. "JBQ" Queru
Software Engineer, Android Open-Source Project, Google.

Questions sent directly to me that have no reason for being private
will likely get ignored or forwarded to a public forum with no further
warning.

[Shachar Shemesh]

GodOfDeath wrote:

Sure what I did was I modified TextView.java, which is available under
"mydroid\frameworks\base\core\java\android\widget"
you can look at the part where it checks for Gravity property. Also
you might wana add one utility.java file which takes care of your
number i.e iif its a hebrew no or not, on checking u can set the
string being passed to SetText() function,simple.
  

Hi,

Without delving into the code too much (yet), this seems to me to be the wrong place to do things. It is too high in the class hierarchy to my liking. I think a better place would be either at android.text.Layout or one of its direct children.

The bad news is that, looking at the code there, it seems that whoever wrote that code had a too-naive view of what BiDi entails. It seems not to be aware of the multi-level requirements that any moderately complex string requires (i.e. - it doesn't seem to differentiate strong LTR, such as Latin letters, from weak LTR, such as numbers). I am now involved in a project that may require going in and fixing it, so I'm hoping I'll have a chance to do it right in the near future.

<shameless plug>
In the mean while, you can find the first two articles in what will hopefully become a series of about 6 or 8 about BiDi aware programming at http://www.lingnu.com/en/programming.html. This text was specifically written to be read by people who do not speak/read BiDi languages. It even has an RSS so you can find out when I release the next articles in the series.
</shameless plug>

[Jean-Baptiste Queru]

Given that the only requirement so far has been pure LTR, the fact
that there's additional code in place is a free bonus, even if that
code isn't perfect.

Good discussion, though, keep going :)

JBQ

[Shachar Shemesh]

Jean-Baptiste Queru wrote:

Given that the only requirement so far has been pure LTR, the fact
that there's additional code in place is a free bonus, even if that
code isn't perfect.

Good discussion, though, keep going :)

JBQ

I'm not so much worried about the code as I am about the interfaces.

android.text.Layout.Directions seems hackish and not up to the task (not that I have managed to completely understand it). If I understand it correctly, it assumes a simplistic, and rather misleading, model of how BiDi works. Though the documentation does have a vague warning that might be read as a deprecation notice on it, I think a more stern one is in order.

The general interfaces into BiDi, as acceptable across almost all APIs, are:
- Logical to Visual (and vice versa) mapping: an array of indexes that matches the nth logical character with the mth visual character (and vice versa). The first is needed for reordering, the second for translating visual actions (such as a mouse click) to logical position (such as the resulting cursor location).
- BiDi levels: numbers in ascending between 0 and 63 (though numbers beyond 2 are extremely rare) signifying the directionality of the characters. If I understood "Directions" correctly, BiDi level % 2 is Direction (even means LTR, odd mean RTL).

Like I said, I am likely to be working on an implementation soon. The beauty of open source...

[Jean-Baptiste Queru]

I'm not personally an expert in bidi, and I've told our text expert
about this thread so that he can choose to participate if he wishes.

If you believe that it would be better to have no APIs at all instead
of the current ones, it might make sense to deprecate the existing
ones right away to avoid that apps use APIs which you believe aren't
going to work in the long term.

Thanks for bringing your expertise to this discussion,
JBQ

[GodOfDeath]

One question jean, could you please tell me if 2.0 is going to support
BiDi languages?

Also I would like to know if you think my modifications in TextView
widget make any sense, my idea behind it was to provide support for
Bidi languages at the base level widget, but i would like your expert
opinion on the changes I've made.

regards
vikram wanchoo

[Jean-Baptiste Queru]

-the only official version numbers so far are 1.0, 1.1 and 1.5. I
don't know which future version (if any) will be 2.0.

-I have no visibility over whether any future version will have better
bidi support (and I'm not actually an expert in that domain so I'll
refrain from reviewing your changes).

JBQ

[Shachar Shemesh]

GodOfDeath wrote:

One question jean, could you please tell me if 2.0 is going to support
BiDi languages?

Also I would like to know if you think my modifications in TextView
widget make any sense, my idea behind it was to provide support for
Bidi languages at the base level widget, but i would like your expert
opinion on the changes I've made.
  

Can you please post a link to the actual patch?

Thanks,

[GodOfDeath]

Although I read above that you have brought this thread to Bidi
experts notice but is it possible to contact him i.e. if I could send
him my changes to verify, I just want to provide any assistance
regarding Bidi.

[Shachar Shemesh]

Jean-Baptiste Queru wrote:

I'm not personally an expert in bidi, and I've told our text expert
about this thread so that he can choose to participate if he wishes.

If you believe that it would be better to have no APIs at all instead
of the current ones, it might make sense to deprecate the existing
ones right away to avoid that apps use APIs which you believe aren't
going to work in the long term.

  

I'm not sure, yet. Some of the APIs are actually quite good. For example, the Layout text alignment APIs are phrased as "natural" and "reverse", which allows easily switching left and right if the paragraph direction changes. Some required APIs are missing, but I'm hoping it won't be too late to add them later (for example - I would like to see the layout resources having an optional field saying "this is a RTL/LTR edit control, regardless of what the input is). As for the things that are downright not correct - I'll reserve judgment until I have actually had a chance to look more closely at it. I'm entirely new to both Android and Java (C, C++ and assembly are my natural habitat), so things are taking a little time.

A question, though. I noticed that the platform has ICU4C installed on it, but not ICU4J. They both have BiDi processing functions, and the jar for ICU4J is 6MB.

[Jean-Baptiste Queru]

Do you have a link to the patch on review.source.android.com ?

JBQ

[joseph...@rollcomm.com]

Dear Jean,

I ask you in your answer signal it, where, how may be to register from the
correspondence?

I say thank you for your honourific answer!

Best regards,

Joseph

Joseph Vajda | Chief Executive Officer
Rollcomm Worldwide Corporation

Mobile phone: +36 70 317 3767
E-mail: joseph...@rollcomm.com
Fax: +36 1 433 21 82
Skype: Rollcomm3
Web: www.rollcomm.com
............................................................
INTERNET MAIL CONFIDENTIALITY
This E-mail is covered by the Electronic Communications Privacy Act, 18 U.
S. C. 2510-2521 and is legally privileged. This information is confidential
information and is intended only for the use of the individual or entity
named above. If the reader of this message is not the intended recipient,
you are hereby notified that any dissemination, distribution or copying of
this communication is strictly prohibited.

[Milad]

Hi,
I could not see any screenshot you said put it here :( can you give me
link again ?
And it's works in Browser ?
another thing, could you give us a patch what you did ?


thanks

[GodOfDeath]

Hi,

Well there's no way to post screen shots on the forum but if you want
I can mail them to you. Currntly I am working on the mixed text
problem, as sharaon had pointed out mixing arabic text and no's is a
problem so I am trying to look into it once its done I'll submit my
changes..


regards
vikram

[G_man]

OK
Don't forget to let us know. I would really love to test it (if that
is possible) and give you some feedback.

[GodOfDeath]

sure no problem :), one thing has any body any Idea if Mirroring is
supported in Android, like there is support in Symbian for RTL
languages. Any help would be appreciated.

[F1Helper]

Hey everyone,

So I've tried what have been described in the discussion, namely,
plugging in ArabicReshaper into TextVeiw to reshape Arabic characters
so android can display Arabic correctly. Well it worked for a small
hello world app. and some other stuff, like for example the title bar
of the browser when browsing an Arabic website.
However, most importantly, browsing arabic websites, text is still
displayed in the unshaped characters. is the text in the browser
displayed on the screen without passing through textview ?

Kind regards

[Jean-Baptiste Queru]

Indeed, the browser doesn't go through textview for its text display
(except for the active text input). I don't think that textview is the
only public text-drawing API.

JBQ

[F1Helper]

Thanks for the response.

Now I tried do go deeper in the hierarchy tree with the Arabic
reshaping , so now I pipe the text first through the ArabicReshaper
class before feeding it in Canvas. Again, my test application can now
display Arabic correctly, title of the android browser display Arabic
correctly but still web pages display unconnected arabic letters.

Any ideas anyone about which class is used in webkit webcore to draw
text ?

Best Regards

[Shachar Shemesh]

F1Helper wrote:

Thanks for the response.

Now I tried do go deeper in the hierarchy tree with the Arabic
reshaping , so now I pipe the text first through the ArabicReshaper
class before feeding it in Canvas. Again, my test application can now
display Arabic correctly, title of the android browser display Arabic
correctly but still web pages display unconnected arabic letters.

Any ideas anyone about which class is used in webkit webcore to draw
text ?
  

The "correct" location to do and CTL processing would be StaticLayout. Pretty much anyone doing BiDi is required to go through this class.

Technically, people can go through DynamicLayout, but internally, it calls StaticLayout (at least for the current implementation).

The bad news is that retrofitting a shaper on StaticLayout is not likely to be simple. If you manage it, please do share your patches.

[Wasil Eltahir]

Hi,
Now with the release of android 1.6 SDK, I noticed that Arabic has
appeared in supported language list. However, some people who tried
the 1.6 developer platform said that IT DOESNT SUPPORT ARABIC still!!
I dont know if that is true or not, can anyone give me an idea?? is
the next 1.6 update going to support arabic or not??

Thanx

On Jul 29, 10:17 am, GodOfDeath <vikram.wanc...@gmail.com> wrote:
> Hi All,
>
> For last one month I've been trying hard to get thisarabic

> localization to work, byArabicLocalization I mean complete support

> for the language, now my 1.5 cupcake code has following things
>

> 1. When I change the phone language toArabic-Egypt from settings

> application it displays localized string forArabiclanguage.(of
> course one has to add values-ar folder with strings.xml in ever app
> for this).
>
> 2. RTL support forArabicstrings for Label,TextView etc, allarabic
> strings will be Identified and automatically it will display it from
> right to left.
>

> 3. Complete IME forarabictoggles between English also.
>
> 4. All theArabiclanguage strings are Reshaped as perarabicsyntax n

[GodOfDeath]

I think its the same issues as polish and other languages in 1.5
release.


regards

[x2an...@gmail.com]

now this just got out An Arabic Rom for HTC Magic

we could use it to figure out how did they do it, it is based on 1.5

link: http://arabickeyboard.googlecode.com/files/RUU_Sapphire_HTC_Arabic_Etisalat_2.56.494.7_release_signed_NoDriver.exe

regards

[rashid lootah]

There is a leak of a rom, for HTC Magic 32A board, its for Etisalat
the UAE provider it support arabic fully connected words i just got
the rom and test it, its working with all the apps. develped by
Parfield.

:)

[rashid lootah]

The rom posted i got it running on magic, i want to move the arabic
support to donut release i tried my best but no hope,,

[Dianne Hackborn]

Please take discussion of hacking on leaked images elsewhere; we should try to focus this list on legal activities.

Thanks.

On Wed, Oct 14, 2009 at 2:42 PM, rashid lootah <rlo...@gmail.com> wrote:

The rom posted i got it running on magic, i want to move the arabic
support to donut release i tried my best but no hope,,

--
Dianne Hackborn
Android framework engineer
hac...@android.com

Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails.  All such questions should be posted on public forums, where I and others can see and answer them.

[OldGooglers]

Does it work well with the broswer?

On Sep 30, 11:31 pm, rashid lootah <rloo...@gmail.com> wrote:
> There is a leak of a rom, for HTC Magic 32A board, its for Etisalat

> the UAE provider it supportarabicfully connected words i just got