Webview alternatives for displaying RTL languages like Biblical Hebrew

[TJ]

I am trying to use Ankidroid on my new LG Marquee phone (Android 2.3), but it will not display Biblical Hebrew correctly. Apparently, Android 2.3 does not support languages like Hebrew, Arabic, etc.

I have thought of several workarounds, but do not know their feasibility. I am not a programmer, just someone who wants to use the cool Ankidroid program to study Hebrew:

(1) A new plugin with Anki (desktop) that generates images in place of the Hebrew text, so that font display is not an issue. 

(2) An alternative to Android's Webview: would java or something else be able to display RTL languages correctly?

Any suggestions? Thank you.

TJ

[Flavio Lerda]

Hello TJ:

On Wed, Feb 1, 2012 at 4:24 AM, TJ <climbingtoy-...@yahoo.com> wrote:

I am trying to use Ankidroid on my new LG Marquee phone (Android 2.3), but it will not display Biblical Hebrew correctly. Apparently, Android 2.3 does not support languages like Hebrew, Arabic, etc.

I have thought of several workarounds, but do not know their feasibility. I am not a programmer, just someone who wants to use the cool Ankidroid program to study Hebrew:

(1) A new plugin with Anki (desktop) that generates images in place of the Hebrew text, so that font display is not an issue. 

Something similar (in principle) to this has been done for LaTeX to display mathematical formula.

In that case, I believe, you write the code for a particular formula and an image is generated for it and used thereafter.

I have not used LaTeX support with AnkiDroid, so I am not sure if that works out of the box.

 

(2) An alternative to Android's Webview: would java or something else be able to display RTL languages correctly?

I think having our own implementation of RTL text layout and rendering might not be trivial to do.

 

Any suggestions? Thank you.

TJ

--
You received this message because you are subscribed to the Google Groups "AnkiDroid" group.
To view this discussion on the web, visit https://groups.google.com/d/msg/anki-android/-/47msYHH_f7EJ.
To post to this group, send an email to anki-a...@googlegroups.com.
To unsubscribe from this group, send email to anki-android...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/anki-android?hl=en-GB.

--
Flavio Lerda

[iniju]

I think for the latex plugin, the images are generated on the deskop and then synced normally to ankidroid every time we sync.
So the AnkiDroid doesn't do image generation on the fly in regards to Latex.

But if a similar plugin can be developed in the desktop version, that would be a neat solution. I don't have much knowledge on the anki plugins to tell how easy that would be.

Kostas

On Wednesday, 1 February 2012 23:44:35 UTC, Flavio wrote:

Hello TJ:

On Wed, Feb 1, 2012 at 4:24 AM, TJ <climbingtoy-onlineorders@yahoo.com> wrote:

I am trying to use Ankidroid on my new LG Marquee phone (Android 2.3), but it will not display Biblical Hebrew correctly. Apparently, Android 2.3 does not support languages like Hebrew, Arabic, etc.

I have thought of several workarounds, but do not know their feasibility. I am not a programmer, just someone who wants to use the cool Ankidroid program to study Hebrew:

(1) A new plugin with Anki (desktop) that generates images in place of the Hebrew text, so that font display is not an issue. 

Something similar (in principle) to this has been done for LaTeX to display mathematical formula.

In that case, I believe, you write the code for a particular formula and an image is generated for it and used thereafter.

I have not used LaTeX support with AnkiDroid, so I am not sure if that works out of the box.

 

(2) An alternative to Android's Webview: would java or something else be able to display RTL languages correctly?

I think having our own implementation of RTL text layout and rendering might not be trivial to do.

 

Any suggestions? Thank you.

TJ

--
You received this message because you are subscribed to the Google Groups "AnkiDroid" group.
To view this discussion on the web, visit https://groups.google.com/d/msg/anki-android/-/47msYHH_f7EJ.
To post to this group, send an email to anki-a...@googlegroups.com.

To unsubscribe from this group, send email to anki-android+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/anki-android?hl=en-GB.

--
Flavio Lerda

[DennisMe]

I just wrote (copy-pasted together, actually) a small java program
during my lunch break as a proof of concept. It reads the Anki
database directly and converts all Hebrew text into (nicely anti-
aliased) .jpeg pictures which can then be accessed from the media
folder. AFAIK, images can't just be synced over to the Android device
though. You need to copy them manually. I'll see if I can't implement
this as an android app or an add-on to AnkiDroid though. Most of it is
bog-standard Java I just copied and pasted off the web, but I have
never attempted to do an Android app so I might be using stuff that
simply doesn't exist on Android. I'll have to look at that over the
weekend.
No reason why someone familiar with python couldn't do this as a plug
in for Anki (Desktop) using ImageMagick or some such library, its
really very simple.

Dennis

On Mar 9, 10:08 am, iniju <inigo.ald...@gmail.com> wrote:
> I think for the latex plugin, the images are generated on the deskop and
> then synced normally to ankidroid every time we sync.
> So the AnkiDroid doesn't do image generation on the fly in regards to Latex.
>
> But if a similar plugin can be developed in the desktop version, that would
> be a neat solution. I don't have much knowledge on the anki plugins to tell
> how easy that would be.
>
> Kostas
>
>
>
>
>
>
>
> On Wednesday, 1 February 2012 23:44:35 UTC, Flavio wrote:
>
> > Hello TJ:
>

> >> anki-android...@googlegroups.com.

[Flavio Lerda]

Interesting work, Dennis.

Do you have the code somewhere were I can have a look at it?

I would be interested in seeing how you generated the images to evaluate if this is something that could be done on Android.

Of course, it would also be possible to get this done on the desktop, as a plug-in, and then AnkiDroid would not need to know about how the images are generated. The trade-off is that one would not be able to edit the deck in AnkiDroid, but would have to use desktop Anki for that.

-Flavio

[Nicolas Raoul]

Yes, very interesting Dennis!
If it is not crazily huge, the code could be incorporated into
AnkiDroid, to generate images on-the-fly.
Where can we see the code?
Thanks a lot!
Nicolas

[DennisMe]

package textToImage;

import java.awt.Color;
import java.awt.Font;
import java.awt.Graphics2D;
import java.awt.RenderingHints;
import java.awt.font.FontRenderContext;
import java.awt.geom.Rectangle2D;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;

import javax.imageio.ImageIO;

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;


class TextToImage{

public static void main(String[] args) throws IOException,
ClassNotFoundException{

// load the sqlite-JDBC driver using the current class loader
Class.forName("org.sqlite.JDBC");

Connection connection = null;
try
{


// create a database connection
connection = DriverManager.getConnection("jdbc:sqlite:/home/
dennis/.anki/decks/Woordenlijst J.W. Wesselius.anki");
Statement statement = connection.createStatement();
statement.setQueryTimeout(30); // set timeout to 30 sec.

//statement.executeUpdate("drop table if exists person");
//statement.executeUpdate("create table person (id integer,
name string)");
//statement.executeUpdate("insert into person values(1,
'leo')");
//statement.executeUpdate("insert into person values(2,
'yui')");
ResultSet rs = statement.executeQuery("select id, value from
fields where ordinal = 0");
while(rs.next())
{
// read the result set
System.out.println("name = " + rs.getString("value"));


//create String object to be converted to image
String sampleText = rs.getString("value");

//Image file name
Long id = rs.getLong("id");
// String fileName = id.toString();

String fileName = sampleText;

//create a File Object
File newFile = new File("./output/" + fileName + ".jpeg");

//create the font you wish to use
Font font = new Font("Cardo", Font.PLAIN, 50);

//create the FontRenderContext object which helps us to
measure the text
FontRenderContext frc = new FontRenderContext(null, true,
true);

//get the height and width of the text
Rectangle2D bounds = font.getStringBounds(sampleText, frc);
int w = (int) bounds.getWidth();
int h = (int) (bounds.getHeight() * 1.5);

//create a BufferedImage object
BufferedImage image = new BufferedImage(w, h,
BufferedImage.TYPE_INT_RGB);

//calling createGraphics() to get the Graphics2D
Graphics2D g = image.createGraphics();

//set color and other parameters
g.setColor(Color.WHITE);
g.fillRect(0, 0, w, h);
g.setColor(Color.BLACK);
g.setFont(font);
g.setRenderingHint(RenderingHints.KEY_ANTIALIASING, // Anti-
alias!
RenderingHints.VALUE_ANTIALIAS_ON);
g.drawString(sampleText, (float) bounds.getX(), (float) -
bounds.getY());

//releasing resources
g.dispose();

//creating the file
ImageIO.write(image, "jpeg", newFile);
}
}
catch(SQLException e)
{
// if the error message is "out of memory",
// it probably means no database file is found
System.err.println(e.getMessage());
}
finally
{
try
{
if(connection != null)
connection.close();
}
catch(SQLException e)
{
// connection close failed.
System.err.println(e);
}
}
}
}

[DennisMe]

Like I said, its mainly copy-pasted from tutorials and "googled" code
I found on the 'net.

A couple of minutes ago I built my first Android package, I'm a happy
man and about to break open the beer to celebrate!

This is what it looks like:

<code>
package nl.meulensteen.helloAndroid;

import android.app.Activity;
import android.graphics.Typeface;
import android.os.Bundle;
import android.widget.TextView;

public class HelloAndroidActivity extends Activity {
/** Called when the activity is first created. */
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
TextView tv = new TextView(this);
Typeface tf = Typeface.createFromAsset(getAssets(),
"fonts/Cardob101.ttf");
tv.setTypeface(tf);
tv.setTextSize(0, 30);
tv.setText("Hello, בִּלְעֲדֵי");
setContentView(tv);

}
}
</code>

The crazy thing is, I tried this simple code (adapted from yet another
tutorial) and, to my surprise, it just works!
The vowel alignment isn't perfect, but it looks very good to me and
shouldn't hinder learning.
Maybe you could have certain fields rendered, not by the html
renderer, but by an (overlaid) Textview?

I don't think my first approach will work on Android, particularly
since there is hardly any of the "awt" stuff available.
But the second approach is a proven concept to me. I tested this all
on a non-modified HTC Legend BTW on Android 2.2.

It would be great if one of you could somehow adapt this to Ankidroid,
because it will be a while before I would dare to take up such a task.
I'm only just beyond the Hello World stage!

[DennisMe]

Ihave Eclipse and all that set up and I can build and deploy AnkiDroid
(latest version from today, following the git instructions), but I
can't get it to pick up any deck (either public or private). It
redirects me to the ankiweb page to log in and then I get a white
screen with a back button. Afterwards it appears to be logged in but
sync sees nothing to do. Do I need to add my decks to some hidden demo
environment?
Is there some other git repo or some kind of "tag" I could use to get
a working copy to try and patch the Hebrew formatting problems? (I'm
more of an SVN user myself, this Got stuff I find quite complicated)

Can anyone help me out?

[Nicolas Raoul]

Hi Dennis,

I am also using the latest Git, and having no particular problem. I
just tried logging out and logging in again, it worked...
Do you see the screen "Sync account" asking for Username and Password
at any point?
If you have a webcam, could you please record AnkiDroid before and
during the redirect?

If you want to avoid Git problems, you can always download as a ZIP:
https://github.com/nicolas-raoul/Anki-Android/zipball/master
and send us a diff (or zip of the src directory)

Thanks for your help!
Nicolas Raoul

[Nicolas Raoul]

Hi Dennis,

I watched the video. Please try clicking "Account synchronieseren"
instead of "Publieke leersets ophalen".
Is it better?

Maybe the button labels are not clear enough?
If even a developer was mistaken, I guess other users will do, so we
must found a way to make the menu items more explicit.

Cheers!
Nicolas Raoul

On Wed, Apr 18, 2012 at 4:41 PM, Nicolas Raoul <nicola...@gmail.com> wrote:
> Hi Dennis!
> Could you please send this to the mailing list instead? or even
> better, create an issue below:
> http://code.google.com/p/ankidroid/issues/
> Thanks for your feedback!
> Nicolas
>
> On Wed, Apr 18, 2012 at 4:28 PM, Dennis Meulensteen
> <den...@meulensteen.nl> wrote:
>> Thanks for helping out Nicolas,
>>
>> Here's a screencast I did using "androidscreencast". I could do one in English, but I suppose I'd have to change some settings in my phone. I have only had this phone for less than a week (my first smart phone and I love it!).
>>
>> I do want to use Git, even though its counter-intuitive to me, so I can keep up-to date with any changes to minimise any patching that may be required later on. I'll see if I can get the Eclipse integration to work.
>>
>> I'm happy to help with a good cause like Anki(droid), maybe I'll stick around.
>>
>> Dennis

[Nicolas Raoul]

Hi Dennis,

> Maybe you could have certain fields rendered, not by the html
> renderer, but by an (overlaid) Textview?

Interesting idea!
If you could find how to do this that would help us a lot.

Cheers!
Nicolas

[DennisMe]

I've been looking at what is actually going wrong. I'm a little
confused though.

When Idisplay cards containing just hebrew, they are shown in reverse
as reported above. (I have a Cardo font on my SD card because that
appears to fix the vowel misplacement issue)
However, when I show cards containing fields made with a mix of hebrew
and western fonts, the order and vowel placement are both correct.
This looks to be a handy work-around, just add some western
characters, but it may also hint at a do-able solution.
I'll keep looking but I'm kind of stumped as to why this is for now...

Dennis

[DennisMe]

Ok, I got it.

The patch attached below fixes the Hebrew formatting for me, in
combination with the installation of the Open Source Cardo font.
There are some caveats though!

1) You need to enable the "Force hebrew RTL" option from the settings
menu.
2) you need to install the Cardo font (see the manual) [may work with
others, just not with the default fonts]
3) The fields containing Hebrew text must all be set to use the Cardo
font
4) Sync in AnkiDroid will not notice font changes so you need to pick
up the deck again and rename the old one in Ankidroid (I don't know
how to delete them) if you change the fonts.
4) Do not use dir="RTL" in your HTML or you will find yourself at the
mercy of the W3C and the unicode consortium:
http://www.w3.org/TR/html401/struct/dirlang.html#h-8.2.4
To sum it up, they don't play nice together...

If I notice any other Biblical Hebrew glitches I'll provide a new
patch.
Feel free to contact me for help on this and related issues if needed.
I can't guarantee I'll be able to find the time, but I will try.

Regards,
Dennis

----------------------------------------------
--- source/nicolas-raoul-Anki-Android-f65d5a1/src/com/ichi2/anki/
Reviewer.java 2012-03-10 08:46:48.000000000 +0100
+++ source/Ankidroid/nicolas-raoul-Anki-Android-f65d5a1/src/com/ichi2/
anki/Reviewer.java 2012-04-18 15:27:28.000000000 +0200
@@ -3016,8 +3016,8 @@
String hebrewText = m.group();
// Some processing before we reverse the Hebrew text
// 1. Remove all Hebrew vowels as they cannot be
displayed properly
- Matcher mv = sHebrewVowelsPattern.matcher(hebrewText);
- hebrewText = mv.replaceAll("");
+ // Matcher mv = sHebrewVowelsPattern.matcher(hebrewText);
+ // hebrewText = mv.replaceAll("");
// 2. Flip open parentheses, brackets and curly brackets
with closed ones and vice-versa
// Matcher mp = sBracketsPattern.matcher(hebrewText);
// StringBuffer sbg = new StringBuffer();
@@ -3051,7 +3051,8 @@
// Log.i(AnkiDroidApp.TAG, "LTR numerals: " +
sbg.codePointAt(i));
// }
// hebrewText = sbg.toString();//reverse().toString();
- m.appendReplacement(sb, hebrewText);
+ String reversed = new StringBuffer
(hebrewText).reverse().toString();
+ m.appendReplacement(sb, reversed );
}
m.appendTail(sb);
return sb.toString();

[Nicolas Raoul]

Great!

For people who don't have the Cardo font, does this make things worse,
or no change?
If it breaks anything, maybe we should check whether the Cardo font is
present or not, before doing the trick?
What do you think?

Cheers!
Nicolas Raoul

[den...@meulensteen.nl]

I guess it would re-introduce the vowel displacement issue as that is caused by the font. It only affects those who explicitly set the hebrew RTL option.

Imho, removing vowels was just a hack waiting for a real solution though. Considering cardo is open source and freely available, I dont think it would be a problem.
People who dont want vowels should not be using them anyway.

[DennisMe]

I just found out that Cardo is a Google web font, so it can even be
included via some CSS custom font encoding:

<link href='http://fonts.googleapis.com/css?family=Cardo'
rel='stylesheet' type='text/css'>

<p style=' font-family: Cardo; line-height:3em; font-size:
40px;'>אַרְבַּע עֶשְׂרֵ</p>

[DennisMe]

I hardly believe it but the patch stopped working sometime today!
I have no idea what happened in between, but somehow bidirectional
text is stuffed again.
You have no idea how this p1sses me off but it looks like I will need
to find out when the bidirectional text algorithm kicks in and when it
doesn't, so I can find out how to manipulate it logically. It may have
something to do with my text input, maybe there are some special
characters in there. I'm investigating and will let you know.
Dennis

[DennisMe]

It appears there is a bidirectional algorithm somewhere in Android,
that is triggering on cards containing both LTR and RTLtext (Excluding
any added html!). Sounds logical so far. Stock AnkiDroid handles these
reasonably well using the cardo font.

Cards with only RTL text are displayed in reverse.It appears that my
patch works fine for plain RTL cards.So I will now concentrate on
finding a way to handle bidirectional text consistently, even though I
tested it succesfully yesterday.
I'll keep digging and report my findings here, but it looks like it
might take much longer than I had hoped.

Dennis

[DennisMe]

I have a question.

Would it be OK to add a custom resource Hebrew font to Anki, which I
could manually kern and optimize for Androids flakey rendering?
This is because the way Android responds to Right to left text appears
in some obsure way to be dependent on the font used! I could optimize
Cardo for use with Biblical Hebrew and force a font substitution as
well as RTL display.
Of course, this wouldn't be much help to learners of modern Hebrew,
but I wouldn't mind doing the same for them though. That would require
a modification to the option "Force RTL for Hebrew" so the user could
choose a modern or Biblical font (these fonts are very different, more
so than say, sans and serif!).
What do you think?

[Nicolas Raoul]

Embedding fonts would make the APK big, which is a big problem for
people with lower-end phones.
How about downloading the font at first use?
The font could be at http://code.google.com/p/ankidroid/downloads

Thanks!
Nicolas Raoul

[iniju]

So, the font that is used affects how android will display Hebrew fonts?

That's useful insight.

If you've established that a certain font works with Biblical Hebrew, then let's go for it.

We can change the option in the Preferences to explicitly download that font and include an override for the card content.

I believe, for non biblical hebrew (no vowel marks), android handles it without problem (maybe still depends on the font though) as long as no RTL markers are used.

So we only need this override for biblical hebrew, right?

Cheers,

Kostas

> To unsubscribe from this group, send email to anki-android+unsubscribe@googlegroups.com.

[den...@meulensteen.nl]

Yea, fonts have an effect. Im just not sure about the logic (if any) behind what happens.
It appears that switching fonts damages the already unstable BIDI algorithm. This means that a font that worked fine before, will suddenly fail. Because I cant see, from code, when the algo is going to screw up or not, I need to find an alternative approach.

I think it would be best if I made a custom-built font and mapped only the Hebrew characters into the LTR font space to bypass the BIDI algo.
Downloading this small file on first use, if not on the SD, seems like an excellent solution. Ill now investigate if this approach will work and let you know.

----- Reply message -----
Van: "iniju" <inigo....@gmail.com>
Datum: zo, apr. 22, 2012 13:00
Onderwerp: [Ankidroid] Re: Webview alternatives for displaying RTL languages like Biblical Hebrew
Aan: <anki-a...@googlegroups.com>

> anki-android...@googlegroups.com.

> > For more options, visit this group at
> http://groups.google.com/group/anki-android?hl=en-GB.
> >
>
>

--
You received this message because you are subscribed to the Google Groups "AnkiDroid" group.

To view this discussion on the web, visit https://groups.google.com/d/msg/anki-android/-/QPR5nXzvLFoJ.

To post to this group, send an email to anki-a...@googlegroups.com.

To unsubscribe from this group, send email to anki-android...@googlegroups.com.

[DennisMe]

Small status update for those interested, and a question or two.

I have now made a custom-built font (based on Cardo) and mapped the

Hebrew characters into the LTR font space to bypass the BIDI algo.

This works well with the cards I have and the ones I downloaded.

What it does is, well nothing.Unless you activate the "Force RTL for
hebrew" option. In that case it will detect hebrew portions of the
text (using the already existing code), reverse consecutive Hebrew
characters and punctuation for RTL display (more or less existing
code) and then wrap the Hebrew in a <span> to apply the custom font
only to the actual Hebrew portions.Then it will translate the hebrew
into the range where there are normally only LTR 'code-points' so the
BiDi algorithm is tricked into submission.
This isn't fool-proof, but should work for 99% of all flash cards.

Now I have proven the concept for myself I'll look into downloading
the custom font on demand.I will also need to tweak the font because
Android does not handle all the formatting which the original author
put into the font. In 99.9% of all words this is no issue, but it can
sometimes lead to confusing effects (like a 'Dagesh' on the wrong side
of a Vav). I'm sure I will find a way to fix this though.

I have a couple of questions for the devs.
To which version should I add the changes? I'm currently on the
Android Market version, but that is for fixes only. Besides, that way
I got myself a working Anki in the fastest possible way ;-) If this is
regarded as a fix, I'm fine with that. OTOH, downloading custom fonts
and such seems more like new functionality so you may want it coded
onto the current alpha to take advantage of the in-depth testing?

What to do with Modern Hebrew. Apparently, as long as there are no
vowel marks and such, MH is treated properly by the BiDi algo.Normally
there won't be vowels in MH, but they ARE still used for teaching the
language to beginners AFAIK. Using a Biblical font for MH must look
Stone-Age, particularly to the more secular audience.It doesn't bother
me but then, I'm studying the language from a religious perspective,
so I am more than a little biased.If a user of MH activates the fix I
made, all the words will suddenly look archaic.
I could provide a second, modern looking font for MH, but We'd need
another option. Or should the current option be turned into some sort
of combobox? Any ideas on how to tackle this? I could also revert to
the original "dump the vowels" approach, but that is rather harsh.
After all, If someone is studying cards with vowels, they must have
been put in there for a reason.

Thanks, Dennis

[Robert Siemer]

Hi,

where is the bug report on Android BIDI? Which versions are affected? Is
there a test webpage to show the problem?

Robert

[Dennis Meulensteen]

Go ahead... Make My Day!

[Robert Siemer]

I made a little test page:

http://backsla.sh/bidi-forced-rtl

I�ve some Hebrew in there, but probably not the Unicode range in
question (Cardo on google.com/webfonts doesn�t show these). Can you
provide an accurate Unicode string �binary� which renders wrong? Or the
full page given to WebKit?

That test page renders fine on my desktop (Chromium), in the Android
standard browser and the �DebugBrowser� on my Android 2.3.7 phone.

Happy?

Robert

[Nicolas Raoul]

Hi Dennis!

I have no experience with RTL, I guess all Hebrew/Arabic users should
test your branch, so that a solution that works for everybody is found
:-)

About the branch:
If you don't feel like switching to 2.0 now (because it is very
unstable), the best would be to create a "cardo" branch starting from
the 1.1 maintenance branch. When you switch to using the 2.0 alphas,
be sure to regularly pull recent changes from master to "cardo", so
that other alpha users can use it.
Haha maybe my explanation is not very clear... if you have any doubt
just ask here.

Cheers!
Nicolas Raoul

[iniju]

Hi Robert,

The problem in question is with Hebrew text that uses vowels marks (obviously not modern hebrew).

An easy way to see the problem is to visit this page on the Android cellphone (use stock browser):

Hebrew_alphabet

First sentence where it says:  The Hebrew alphabet (Hebrew: אָלֶף־בֵּית עִבְרִי ...

The Hebrew text there contains vowel marks and although my desktop browser (Chrome) renders that as expected, on the cellphone, the vowels are displaced to the left of the corresponding consonant.

Apologies if my terminology is weird, I don't know any Hebrew.

Kind regards

Kostas

[iniju]

Hi Dennis,

Your solution is very clever, thank you very much for your work on this. I never thought of modifying the font in order to trick android.

As you say I think we should make the RTL override work only for archaic hebrew text, since modern hebrew renders fine. We'll change the description of the option to clearly say that.

I don't think there will be any problem integrating your changes to the new version of AnkiDroid, since the Hebrew-Fix code is independent from the anki desktop and as such untouched by the upgrade.

Nicolas's suggestion of a new branch off of the market version seems the cleaner way to contribute your change.

Then we can merge it on the 2.0 branch.

We still have to find a way to make your modified font available to users. Host it on the downloads page?

What about the licence? SIL Open Font License seems quite liberal. 

Cheers,

Kostas

[Nicolas Raoul]

> We still have to find a way to make your modified font available to users.
> Host it on the downloads page?

Indeed, we already store a few fonts there, and also a file that is
loaded by all AnkiDroid instances. That would be the most logical
place.
http://code.google.com/p/ankidroid/downloads

Cheers!
Nicolas Raoul

> --
> You received this message because you are subscribed to the Google Groups
> "AnkiDroid" group.
> To view this discussion on the web, visit

> https://groups.google.com/d/msg/anki-android/-/VlBVvnOVSXgJ.

[iniju]

The related bug report on Android:

http://code.google.com/p/android/issues/detail?id=14661 

According to the last comment, the problem is fixed on ICS, but that doesn't help us much.

I can give it a try at home from the tablet.

Cheers,

Kostas

[Robert Siemer]

On 25/04/12 17:02, iniju wrote:
> The related bug report on Android:
> http://code.google.com/p/android/issues/detail?id=14661
>
> According to the last comment, the problem is fixed on ICS, but that
> doesn't help us much.
>
> I can give it a try at home from the tablet.

I added two comments and six photos/screenshot on that bug report.

I did a couple of test with interesting results: On �broken� devices,
any character composition just doens�t work (and results in text going
always left-to-right). � The �fixed� devices (which there are from 2.2.x
on, but not in vanilla Android up to ?.?.?) just know better about
Hebrew with vowels going RTL. The other compositions still don�t work.

Leaves open the questions:
-Which vanilla Android fixed Hebrew RTL composition?
-Which vanilla Android fixes composition in general? Can someone check a
ICS 4.x version?

Dennis, which Android(s) are you running?


Robert

> --
> You received this message because you are subscribed to the Google
> Groups "AnkiDroid" group.
> To view this discussion on the web, visit

> https://groups.google.com/d/msg/anki-android/-/7SLhVpTJlycJ.

[Dennis Meulensteen]

I'm running 2.2.

Most of my discoveries are spread out on this thread.

What I noticed was that

- Hebrew vowel positioning seems to be dependent mostly on the font. Cardo works reasonably, most others don't. This is apart from the RTL vs. LTR issue.

- Font mark to base alignment functions do not seem to be implemented, GPOS tables related to anchor positioning are one of them. This means that special formatting is difficult to achieve.

- on my machine hebrew was only detected properly in some cases and this was rather unpredictable. It seemed like switching to another font sometimes reversed the ability of whatever bidirectional text algorithm there is in Android to detect Hebrew characters. I was unable to determine the exact logic behind this though.

- the algorithm didn't seem to take any notice of HTML markup like dir= , <LTR> <LRO> etc.
- the algorithm didn't seem to take any notice of any of the unicode bidirectional override characters.

Thanks for picking up the ball Robert, very sportsmanlike.

Dennis

On Wednesday 25 April 2012 20:06:04 Robert Siemer wrote:
>
> On 25/04/12 17:02, iniju wrote:
> > The related bug report on Android:
> > http://code.google.com/p/android/issues/detail?id=14661
> >
> > According to the last comment, the problem is fixed on ICS, but that
> > doesn't help us much.
> >
> > I can give it a try at home from the tablet.
>
>
> I added two comments and six photos/screenshot on that bug report.
>

> I did a couple of test with interesting results: On “broken” devices,
> any character composition just doens’t work (and results in text going
> always left-to-right). – The “fixed” devices (which there are from 2.2.x

> on, but not in vanilla Android up to ?.?.?) just know better about

> Hebrew with vowels going RTL. The other compositions still don’t work.

[Antoine Schweitzer-Chaput]

> Leaves open the questions:
> -Which vanilla Android fixed Hebrew RTL composition?
> -Which vanilla Android fixes composition in general? Can someone check a ICS 4.x version?

I don't know about Hebrew, but Arabic was fixed in ICS.
All previous vanilla versions had RTL issues with Arabic. RTL was performed correctly in some cases, but several patterns broke it and forced LTR.
These patterns involved mostly diacritics (i.e. vowels), but also some other specific letters or sequences.
The patterns I could find are included in AnkiDroid since 1.1, they were found mostly by trial-and-error.

[Robert Siemer]

On 26/04/12 03:22, Dennis Meulensteen wrote:
>
> I'm running 2.2.
>
> Most of my discoveries are spread out on this thread.
>
> What I noticed was that
>
> - Hebrew vowel positioning seems to be dependent mostly on the font.
> Cardo works reasonably, most others don't. This is apart from the RTL
> vs. LTR issue.

The Cardo font on google.com/webfonts puts xxxxx when I write Hebrew
example text there. � Is that the same font? Does it behave different
when you embed it in your own pages??

I would add the font to my test page. I would be VERY curious to see
screen-shots of the same stuff rendering different depending on the font
and time of the day... � Information missing in the font could be fatal,
sure, but that it sometimes works and sometimes not, is not really
something I believe in...

> - Font mark to base alignment functions do not seem to be
> implemented, GPOS tables related to anchor positioning are one of
> them. This means that special formatting is difficult to achieve.
>
> - on my machine hebrew was only detected properly in some cases and
> this was rather unpredictable. It seemed like switching to another
> font sometimes reversed the ability of whatever bidirectional text
> algorithm there is in Android to detect Hebrew characters. I was
> unable to determine the exact logic behind this though.

Can you add a screen-shot of my test page on that google bug?

> - the algorithm didn't seem to take any notice of HTML markup like
> dir= ,<LTR> <LRO> etc.

dir= has different meanings depending on the element. <ltr>, <lro> don�t
exist.

> - the algorithm didn't seem to take any
> notice of any of the unicode bidirectional override characters.
>
> Thanks for picking up the ball Robert, very sportsmanlike.

Not sure how to take this... ;-)


Robert

> On Wednesday 25 April 2012 20:06:04 Robert Siemer wrote:
>>
>> On 25/04/12 17:02, iniju wrote:
>>> The related bug report on Android:
>>> http://code.google.com/p/android/issues/detail?id=14661
>>>
>>> According to the last comment, the problem is fixed on ICS, but
>>> that doesn't help us much.
>>>
>>> I can give it a try at home from the tablet.
>>
>>
>> I added two comments and six photos/screenshot on that bug report.
>>

>> I did a couple of test with interesting results: On �broken�
>> devices, any character composition just doens�t work (and results
>> in text going always left-to-right). � The �fixed� devices (which

>> there are from 2.2.x on, but not in vanilla Android up to ?.?.?)
>> just know better about Hebrew with vowels going RTL. The other

>> compositions still don�t work.

[Robert Siemer]

On 26/04/12 03:34, Antoine Schweitzer-Chaput wrote:
>> Leaves open the questions:
>> -Which vanilla Android fixed Hebrew RTL composition?
>> -Which vanilla Android fixes composition in general? Can someone check a ICS 4.x version?
>
> I don't know about Hebrew, but Arabic was fixed in ICS.

I just saw that on that google bug report that ICS renders those
Hebrew+vowel examples worse then my 2.3.7! � Doesn�t look like fixed to
me... � Is Arabic with vowels perfect on it??

> All previous vanilla versions had RTL issues with Arabic. RTL was performed correctly in some cases, but several patterns broke it and forced LTR.
> These patterns involved mostly diacritics (i.e. vowels), but also some other specific letters or sequences.
> The patterns I could find are included in AnkiDroid since 1.1, they were found mostly by trial-and-error.

I put it on my TODO list to add Arabic on my test page. � I would be
faster (but don�t hold your breath) if you give me 1) a webpage which
renders wrong on pre-ICS (instead of letting me fish it out of the
source :-), and nice would be 2) a screen-shot of the mistakes. � Is
there a bug-report with that for Arabic (screen-shots and/or link)?

I�m almost sure CyanogenMod has usable (pre-ICS) Arabic support.


Robert

[Antoine Schweitzer-Chaput]

> I just saw that on that google bug report that ICS renders those Hebrew+vowel examples worse then my 2.3.7! – Doesn’t look like fixed to me... – Is Arabic with vowels perfect on it??

Not quite perfect, vowels sometimes overlap with the regular letters, but I haven't encountered any RTL or reshaping issue since upgrading.

> I put it on my TODO list to add Arabic on my test page. – I would be faster (but don’t hold your breath) if you give me 1) a webpage which renders wrong on pre-ICS (instead of letting me fish it out of the source :-), and nice would be 2) a screen-shot of the mistakes. – Is there a bug-report with that for Arabic (screen-shots and/or link)?

Actually, pre-ICS Arabic display is completely messed up because no reshaping is performed *and* RTL is wrong.
The following page shows some screenshots:
http://www.linuxac.org/forum/showthread.php/49489-%D8%A7%D9%84%D8%BA%D9%87-%D8%A7%D9%84%D8%B9%D8%B1%D8%A8%D9%8A%D8%A9-%D9%85%D8%AF%D8%B9%D9%88%D9%85%D9%87-%D9%81%D8%B9%D9%84%D9%8A%D8%A7%D9%8B-%D9%81%D9%89-Android-2-3-Gingerbread

> I’m almost sure CyanogenMod has usable (pre-ICS) Arabic support.

Right.
A patch exists to make arabic usable on pre-ICS versions of Android. It was included in CyanogenMod and also on custom Android versions shipped by some phone manufacturers.
It was however never accepted by Google because it didn't match with the semantics of the existing APIs, which is why they reworked this part extensively for ICS.

[Robert Siemer]

On 26/04/12 06:40, Antoine Schweitzer-Chaput wrote:
>
>> I just saw that on that google bug report that ICS renders those

>> Hebrew+vowel examples worse then my 2.3.7! � Doesn�t look like
>> fixed to me... � Is Arabic with vowels perfect on it??

>
> Not quite perfect, vowels sometimes overlap with the regular letters,
> but I haven't encountered any RTL or reshaping issue since
> upgrading.

Can you go to

http://backsla.sh/bidi-forced-rtl

with Android 4 and compare with my latest screen-shot on the linked bug
report? Can Android 4 compose Arabic (NFD) as good as my patched Android
2.3.7? Better? Worse?

> The following page shows some screenshots:
> http://www.linuxac.org/forum/showthread.php/49489-%D8%A7%D9%84%D8%BA%D9%87-%D8%A7%D9%84%D8%B9%D8%B1%D8%A8%D9%8A%D8%A9-%D9%85%D8%AF%D8%B9%D9%88%D9%85%D9%87-%D9%81%D8%B9%D9%84%D9%8A%D8%A7%D9%8B-%D9%81%D9%89-Android-2-3-Gingerbread

I picked some random text snippets for my test page from there where I
found pre-combined characters.



>> I�m almost sure CyanogenMod has usable (pre-ICS) Arabic support.

>
> Right. A patch exists to make arabic usable on pre-ICS versions of
> Android. It was included in CyanogenMod and also on custom Android
> versions shipped by some phone manufacturers. It was however never
> accepted by Google because it didn't match with the semantics of the
> existing APIs, which is why they reworked this part extensively for
> ICS.

Which API?


Robert

[ant...@schweitzer-chaput.fr]

> Can you go to

> http://backsla.sh/bidi-forced-rtl
> with Android 4 and compare with my latest screen-shot on the linked bug
> report? Can Android 4 compose Arabic (NFD) as good as my patched Android
> 2.3.7? Better? Worse?

 

Attached is the screenshot, taken with a Nexus S running (unpatched) ICS 4.0.4.

As you can see it is very similar to your latest one.

The main difference I see is with the vowels placement, which is (in my opinion) too far up or down on your screenshot.

I guess this is mostly related to the font, and it's probably mostly a matter of taste. 

 

> Which API?

 

The best explanation I could find is in this thread: 

http://groups.google.com/group/android-platform/browse_thread/thread/664c4331a9c69b72/691e4f4f3b6953e3

In particular the message from Shachar Shemesh on December 12th 2010. 

[Robert Siemer]

As Dennis mentioned, the font plays a role! � I added the Cardo font to
my test page (and screen shots on the linked bug report).

-the font changes whether combining works and how it looks in the end on
my desktop (useless ltr Hebrew with vowels looks shifted) and my mobile
phone (enables Latin combining)

-the font does not influence ltr/rtl directions on my desktop (still all
fine) and mobile (Latin _with_ combining characters still doesn�t go RTL)

> > http://backsla.sh/bidi-forced-rtl
> > with Android 4 and compare with my latest screen-shot on the linked bug
> > report? Can Android 4 compose Arabic (NFD) as good as my patched Android
> > 2.3.7? Better? Worse?
>
> Attached is the screenshot, taken with a Nexus S running (unpatched) ICS
> 4.0.4.
>
> As you can see it is very similar to your latest one.
>
> The main difference I see is with the vowels placement, which is (in my
> opinion) too far up or down on your screenshot.

But interestingly not too far right or left as Hebrew vowels on ICS.

Does the new Cardo font on my test page make a difference for Hebrew? I
guess it may help...

> I guess this is mostly related to the font, and it's probably mostly a
> matter of taste.

Eh... yes. ;-)

> > Which API?
>
> The best explanation I could find is in this thread:
>
> http://groups.google.com/group/android-platform/browse_thread/thread/664c4331a9c69b72/691e4f4f3b6953e3

Interesting thread. But I don�t see any big changes to the API in ICS
here. They made a documentation change: removed the sentences �doesn�t
do BiDi� from drawText()?



Robert