Some Points regarding the project "Word level Highlighting"

[Yash Girdhar]

 I have been working on the Project "Word level highlighting" and want to put down some points:

As I have understood from the flite tts code,

->NativeFliteTTS.java interacts with the flite library
->fliteService.cpp receives all the actions from the the java code and then, appropriate actions are taken in the fliteEngine.cpp

Now, we are able to get callbacks in the fliteEngine.cpp after each word and receive the same in NativeFliteTTS.java.
(as described  in the image above)

If I have correctly understood the problem, we have to send these same values to the GoRead app.

The solution I propose for this problem is:

1.porting the flite library into the app

Flite calculates everything on itself for text to speech, but uses "android_tts_synth_cb_t ttsSynthDoneCBPointer" to output the voice.
This is the only part of android, that is uses, which prevents it from being an independent library and installs it as android-speech service.

So, what we can do is, we can handle this corresponding library accordingly / find an alternate for it, and then, can port this along with the flite library,to the project.
I am searching for the same, and will update if I find something.

2.Secondly, we are getting the values in the java code of flite tts.
So, can we use public intents (or something like that), to send these values to the goread?
We can send data across the apps in android, but i am not sure if we an do it here? Acc. to me, this option can be considered if it can be precisely shaped according to our need.  
I will try to test the second option on a test app provide you with the observations.

Also, if we port the Flite tts to the app, we would also have to change the highlighting and tts part accordingly.
But the main part is getting the callback values in the app,


Please provide feedback on this, so that I can continue in the right direction.

Cheers
yashasvi

  

[Rom Srinivasan]

Hello Yash,

We are thinking about essentially bundling the NativeFliteTTS java code along with the native C++ components into the Go Read app. We would then add a user setting that lets a user choose between Flite or the default Android TTS. Only if they choose Flite will they get the benefit of the Flite word level callbacks. In this case, depending on the user's choice of TTS engine, we would have an alternate code path for the highlighting.

But since our initial plan is to bundle Flite with Go Read at the source level, we shouldn't need to use any intents or broadcasts. That would make sense if we wanted to make the Flite TTS a standalone TTS that could interact with other apps.

 

[Yash Girdhar]

 its good to hear the feature of 'providing choice to the reader' of choosing the tts.
 
In that case, as I mentioned above, fliteTTS is doing everything on its own, to synthesize the text (eg. word durations, segment durations, indices etc.)
 , except for the one thing .android_tts_synth_cb_t , that's preventing it to be independently ported to goread.

So,I guess we would have to find a solution to this, by searching some alternative or modifying its own library.

I am working on this and will update you if I am able to find something.

[Kapil Dontula]

Hello,

I think even using Flite as a standalone app can allow people to choose the TTS. If they choose flite as their tts app intent can be used for word level feed back else normal pico tts will do the job. This solution looks like a more general solution rather than bundling flite and goread. As to bundle flite and goread together we need to find the source implementation of the method "android_tts_synth_cb_t". which helps the tts app to read the text by taking input of all prasodic features along with the text.

Thanks,

Dontula Kapil.

[Yash Girdhar]

  After considering both the solutions, I finally arrived at the following:

After going through the whole implementation of flite, I think I have figured out to do this, without interfering with the "android_tts_synth_cb_t";

first, we will combine the both apps (Java part) ,(so as to have a single apk). Then, at the init() time, we will pass a callback function from java into the C code.Then modify the flite callback function(that calls tts_synth_cb_t) to call our java funtion.If we combine the two, this shouldn't be a problem.

One more thing, I figured out that the word level callback that were printed in the logcat in the last year project, were from the C file only.
So, I started with sending these callbacks to the java of flite itself, and I have been successful in doing so, and have checked the same by printing in the java files.
I wanted to attach the apk also. but I am stuck in a threading error, for the last whole day.

Now, what I propose is:

When we are combining the two apps ( java code), we can send this callback to the java code of GoRead also .(we are already getting the callback in the flite java, and now, as the package would be same, this can be done).

Right now, I am trying this on a test app, and will post the update, in a short time.
If this works on the test app, this should definitely work in goread and according to me, there is the best solution we can have.
Also, It automatically installs the tts also on the device, compare to other methods (porting the flite iso to the android). 

Rom sir, Please provide your opinion on this.If this option seems feasible to you, I will start writing the detailed description of this in my proposal.

Cheers
Yashasvi

[Rom Srinivasan]

Yash, that sounds like the solution we are looking for!
(without the threading error ;)

Also, I just wanted to make sure you are using Kapil's latest test code at https://github.com/dontulakapil/tts-test-app . Kapil can confirm if that is, in fact, the latest.

[Yash Girdhar]

Sure sir, I'll confirm with the Kapil sir.

I have written my proposal for the project, that contains the detailed explanation of the solution I am proposing.
It would be great if you could just review that ,so that it would help me to figure out if any change is required in any part of the project plan.
Please tell me how can I send you the same.

Cheers 
Yashasvi

[Rom Srinivasan]

Yash,

Please apply on the GSOC site and I'll take a look at it and give you feedback. The site lets you modify your application and proposal and project plan until the final due date.

[Ankit Aggarwal]

Rom,

Could you please go over my proposal regarding Tecla accessiblity clubbed with another idea and provide feedback? I have submitted it officially through the gsoc site.

And who would be the mentors for these projects?

[Yash Girdhar]

Sir, I have submitted my proposal on the GSOC site.
Please have a look at it.

cheers
yashasvi

[Rom Srinivasan]

Yash,

For some reason the links to your resume and Github profile point to Kapil's resume and Gitub profile. Can you please fix that?

[Yash Girdhar]

Corrected sir 

Sorry for the mistake.
Actually, I was referring to kapil sir's previous work at that time. 

[Ankit Aggarwal]

Sir,

I have updated my .apk file and created a 2nd version showing Tecla accessibility with the latest changes. I have updated my proposal with a link to the new .apk.

I have also submitted a pull request.

Please provide feedback.

Cheers,

Ankit Aggarwal

[Kapil Dontula]

Hi yash,

I think the solution you provided, certainly can solve this problem.Now, it just needs to be implemented correctly.
I went through your proposal for ensuring the same and would like to talk about certain things.
First of all, I appreciate the structure of your proposal, especially how you have divided the development into different phases, along with a structured timeline. Also, its good that you have also highlighted the basic architecture and considerations during the project. 
According to me, the project needs to be strictly followed with the timeline, to be implemented efficiently.

Thanks,

Dontula Kapil,

IIIT Hyderabad.

--
You received this message because you are subscribed to a topic in the Google Groups "Accessible FBReaderJ" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/accessible-fbreaderj/XXZj9s8c-Bs/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to accessible-fbrea...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Yash Girdhar]

Thanks Kapil sir for the feedback. I would also request Rom sir to please provide a feedback about the proposal so that I can make any necessary changes in the proposal before the deadline.

Yash

To unsubscribe from this group and all its topics, send an email to accessible-fbreaderj+unsub...@googlegroups.com.

[Yash Girdhar]

 
 Sir,
    I discussed with Alok parlikar ( who made the flite tts ) , regarding the above solution.
   He agrees with the part that if we combine the two apps, we can send the two callbacks simultaneously, one to android_tts_synth_cb_t, and 
   one to GoRead.That way, both highlighting will happen and playback too.