Android 2.0 Google Maps Navigation and TTS usage for turn-by-turn directions?

[blindfold]

Is anything known about how an app may use the turn-by-turn directions
of Android 2.0 Google Maps Navigation in combination with the Android
TTS APIs? Until now we were prohibited by the Google Maps ToS to offer
turn-by-turn directions. Clearly it would be attractive to integrate
some of that functionality in accessible applications. Or can Google
Maps Navigation run as a service, such that one need not switch
between applications (since Android only runs one app at a time) to
still get spoken turn-by-turn directions along with the app's own - in
my case vision-based - navigation functionality?

My "dream" is to offer something like I outlined back in 2003 in a
Christian Science Monitor interview,

http://www.csmonitor.com/2003/0710/p13s01-stct.html

combining accessible vision-based sound navigation with GPS-based
speech navigation.

Thanks,

Peter Meijer


The vOICe for Android
http://www.seeingwithsound.com/android.htm

[Beth Koenig]

Keep working on this peter! It is so important. if we can find a work
around... I love being able to hear where I am but I normally use the
shell for that. Sonification doesn't help me much because of the
visual problem with my brain but I have trained others to use it. In
fact one total blind woman I worked with in the PC verision of the
vOICe was able to pickout details and show me exactly where on the
screen they are. It was really cool how fast she picked it up. I am
going to be moving soon and I would love to have the ability to get
more detailed real time directions. Right now I'm stuck between the
screen reader and my vision.
Thanks,
Beth Koenig
bet...@gmail.com
Director of Deaf Blind Services
Orange County Deaf Advocacy Center
2960 Main Street A100
Irvine, CA 92614
http://www.deafadvocacy.org/dbs
Health, safety, and productivity are the cornerstones of independence.
At the Orange County Deaf Advocacy Center we provide the training and
services necessary for the deaf and disabled to achieve equality and
independence in all areas of life.

[CLC]

I believe you actually meant "only one app is shown in the foreground
at a time." Android does run multiple applications simultaneously; in
fact, having the ability to run an app in the background is one of the
Android platform's features. Please see:
http://www.youtube.com/watch?v=q1m4j676BCc

As far as the Maps APIs go, please see the API reference here:
http://code.google.com/android/add-ons/google-apis/reference/index.html

Also, a better place to discuss some of these questions might be the
main Android developers list (http://groups.google.com/group/android-
developers) - this way, there are more Android developers who will see
your question and hopefully provide a better answer.

-Charles

P.S. Peter, in your specific case, have you considered using Google
walking directions instead?
http://maps.google.com/maps?f=d&dirflg=w&utm_campaign=en&utm_medium=ha&utm_source=en-ha-na-us-sk-dd&utm_term=Walking

The reason is that the Navigation feature in Android 2.0 Google Maps
is designed for driving, and those directions may not be appropriate
for your use case (ie, imagine telling your users to walk onto a
freeway...).

[blindfold]

Thank you for your answers, Charles! I will consider posting to
android-developers, but had posted here because I expect that Google
Maps Navigation uses the same TTS functionnality for its spoken
directions that is the focus of this group in discussing interactions
with third-party self-voicing applications and system-wide screen
readers. In short, I'd think the most knowledgeable people on this
theme are here.

> Android does run multiple applications simultaneously; in fact,
> having the ability to run an app in the background is one of the
> Android platform's features.

It depends on what one means by "running", but my understanding is
that a normal Android app, by which I here mean an Activity, does not
perform *any* task if not in the foreground, and can be completely
killed by the system at any moment, e.g. when short on resources. It
is essentially in hibernation until brought back to the foreground.
Alive yes, with its state preserved, but not "running" in a functional
sense. That is what I meant by saying that Android runs only one app
at a time, because this would imply that if Google Maps Navigation is
an Activity it would not be giving directions while not in the
foreground, that is, when my own app (an Activity) is running in the
foreground. *Only* when Google Maps Navigation is a *Service* could it
(but *does* it also?) continue to give spoken directions while my app
is running in the foreground.

Now I do not currently know if Google Maps Navigation is an Activity
or a Service, hence my inquiry.

> As far as the Maps APIs go, please see the API reference here:
> http://code.google.com/android/add-ons/google-apis/reference/index.html

http://code.google.com/android/add-ons/google-apis/maps-api-signup.html

reads "The Android Maps APIs explicitly do not include any driving
directions data or local search data that may be owned or licensed by
Google." So Google Maps Navigation goes beyond the Maps APIs by giving
actual driving directions, hence the question if one can tap into that
newly added functionality through some other or extended API, or
whether one can at best hope to have Google Maps Navigation running -
and still speaking - as a Service while one's own app Activity is
running in the foreground.

> P.S. Peter, in your specific case, have you considered using Google
> walking directions instead?

Yes, that will be a useful refinement. I had assumed that one could
now also set this in Google Maps Navigation, such that one will then
have turn-by-turn spoken walking directions, because the Maps ToS
unfortunately do not allow me to implement this direcly on top of the
Android Maps APIs.

Thanks!

Peter

The vOICe for Android
http://www.seeingwithsound.com/android.htm

P.S. Thank you Beth, for your words of support and interesting
supplementary information.

On Oct 29, 5:51 pm, CLC <clchen+...@google.com> wrote:
> I believe you actually meant "only one app is shown in the foreground
> at a time." Android does run multiple applications simultaneously; in
> fact, having the ability to run an app in the background is one of the
> Android platform's features. Please see:http://www.youtube.com/watch?v=q1m4j676BCc
>
> As far as the Maps APIs go, please see the API reference here:http://code.google.com/android/add-ons/google-apis/reference/index.html
>
> Also, a better place to discuss some of these questions might be the
> main Android developers list (http://groups.google.com/group/android-
> developers) - this way, there are more Android developers who will see
> your question and hopefully provide a better answer.
>
> -Charles
>
> P.S. Peter, in your specific case, have you considered using Google

> walking directions instead?http://maps.google.com/maps?f=d&dirflg=w&utm_campaign=en&utm_medium=h...

[CLC]

On Oct 29, 1:48 pm, blindfold <seeingwithso...@gmail.com> wrote:
> Thank you for your answers, Charles! I will consider posting to
> android-developers, but had posted here because I expect that Google
> Maps Navigation uses the same TTS functionnality for its spoken
> directions that is the focus of this group in discussing interactions
> with third-party self-voicing applications and system-wide screen
> readers. In short, I'd think the most knowledgeable people on this
> theme are here.

The Navigation team uses the stable platform version of the
TextToSpeech API, and they do not check this list frequently (if at
all).

[davemac]

Not so! (the claim that Android does not run simultaneous processes).
Android does indeed run multiple processes simultaneously, and I'll
prove it to you. But first, an explanation for the confusion. In
Android, a *well-behaved* application will suspend user interaction
when the onPause() callback is invoked. This appears to the user that
the application has been put to sleep. And yes indeed, an application
in this state could be killed if resources get tight. If your
application is speaking text, and the user gets a phone call, of
course you want the audio to stop while the user is talking on the
phone. But your application doesn't stop running just because some
other app has been given center stage. If you don't believe me, try
out this little demo:

/res/layout/main.xml:
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/
android"
android:orientation="vertical" android:layout_width="fill_parent"
android:layout_height="fill_parent">

<Button android:id="@+id/startStopBtn"
android:text="Start"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:enabled="false"
android:onClick="startStop" />

</LinearLayout>


and now your Java code:

import java.util.HashMap;
import android.app.Activity;
import android.content.Intent;
import android.os.Bundle;
import android.speech.tts.TextToSpeech;
import android.speech.tts.TextToSpeech.OnInitListener;
import android.speech.tts.TextToSpeech.OnUtteranceCompletedListener;
import android.util.Log;
import android.view.View;
import android.widget.Button;

public class MainActivity extends Activity implements OnInitListener,
OnUtteranceCompletedListener {
private int counter = 1;
private Button startStopBtn = null;
private static final int REQ_TTS_STATUS_CHECK = 0;
private static final String TAG = "TTS Demo Background";
private TextToSpeech mTts;
private HashMap<String, String> params = new HashMap<String,
String>();

/** Called when the activity is first created. */
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);

startStopBtn = (Button)findViewById(R.id.startStopBtn);

// Check to be sure that TTS exists and is okay to use
Intent checkIntent = new Intent();
checkIntent.setAction
(TextToSpeech.Engine.ACTION_CHECK_TTS_DATA);
startActivityForResult(checkIntent, REQ_TTS_STATUS_CHECK);
}

public void startStop(View view) {
if(mTts.isSpeaking()) {
startStopBtn.setText("Start");
mTts.stop();
}
else {
mTts.setOnUtteranceCompletedListener(this);
startStopBtn.setText("Stop");
sayNext();
}
};

private void sayNext() {
params.put(TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID,
"value");
mTts.speak(String.valueOf(counter++), TextToSpeech.QUEUE_ADD,
params);
params.put(TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID,
"silence");
mTts.playSilence(2000, TextToSpeech.QUEUE_ADD, params);
}

protected void onActivityResult(int requestCode, int resultCode,
Intent data) {
if (requestCode == REQ_TTS_STATUS_CHECK) {
switch (resultCode) {
case TextToSpeech.Engine.CHECK_VOICE_DATA_PASS:
// TTS is up and running
mTts = new TextToSpeech(this, this);
Log.v(TAG, "Pico is installed okay");
break;
case TextToSpeech.Engine.CHECK_VOICE_DATA_BAD_DATA:
case TextToSpeech.Engine.CHECK_VOICE_DATA_MISSING_DATA:
case TextToSpeech.Engine.CHECK_VOICE_DATA_MISSING_VOLUME:
// missing data, install it
Log.v(TAG, "Need language stuff: " + resultCode);
Intent installIntent = new Intent();
installIntent.setAction(
TextToSpeech.Engine.ACTION_INSTALL_TTS_DATA);
startActivity(installIntent);
break;
case TextToSpeech.Engine.CHECK_VOICE_DATA_FAIL:
default:
Log.e(TAG, "Got a failure. TTS apparently not
available");
}
}
else {
// Got something else
}
}

@Override
public void onInit(int status) {
// Now that the TTS engine is ready, we enable the button
if( status == TextToSpeech.SUCCESS) {
startStopBtn.setEnabled(true);
}
}

@Override
public void onPause()
{
super.onPause();
Log.v(TAG,"In onPause()");
}

@Override
public void onDestroy()
{
super.onDestroy();
// if we're going away, clean up
if( mTts != null) {
mTts.stop();
mTts.shutdown();
}
}

@Override
public void onUtteranceCompleted(String uttId) {
Log.v(TAG,"Got utterance: " + uttId);
if("silence".compareTo(uttId) == 0)
sayNext();
}
}

Run it and click the Start button. Now hit the Home button and start
up some other application. You should continue to hear numbers being
counted to you even though you're now using another application.
Pretty soon I think you'll realize why applications "go to sleep" when
they lose focus. If you change the duration argument of playSilence to
a smaller number than 2000 (100 did it for me), you will find that
Android decides to kill the application a little while after it has
been backgrounded.

- dave

> The vOICe for Androidhttp://www.seeingwithsound.com/android.htm

[blindfold]

Thank you for your clarification, Dave. In that case I hope that
Google Maps Navigation is either an *ill-behaved* application
(Activity) ;-) , or a well-behaved Service, or that it offers
programming hooks for other applications to tap into the turn-by-turn
directions part without violating the Google Maps ToS.

Peter

> ...
>
> read more »

[GregM]

For completeness you should probably do this

else
{
super.onActivityResult(requestCode, resultCode, data);
}

instead of just // Got something else

On Oct 31, 2:37 pm, blindfold <seeingwithso...@gmail.com> wrote:
> Thank you for your clarification, Dave. In that case I hope that
> Google Maps Navigation is either an *ill-behaved* application
> (Activity) ;-) , or a well-behaved Service, or that it offers
> programming hooks for other applications to tap into the turn-by-turn
> directions part without violating the Google Maps ToS.
>
> Peter
>
> On Oct 31, 4:56 pm, davemac <davemac...@gmail.com> wrote:
>
>
>
> > Not so! (the claim that Android does not run simultaneous processes).
> > Android does indeed run multiple processes simultaneously, and I'll
> > prove it to you. But first, an explanation for the confusion. In
> > Android, a *well-behaved* application will suspend user interaction
> > when the onPause() callback is invoked. This appears to the user that
> > the application has been put to sleep. And yes indeed, an application
> > in this state could be killedifresources get tight.Ifyour
> > application is speaking text, and the user gets a phone call, of
> > course you want the audio to stop while the user is talking on the
> > phone. But your application doesn't stop running just because some

> > other app has been given center stage.Ifyou don't believe me, try

> >         //Checkto be sure that TTS exists and is okay to use

> >         Intent checkIntent = new Intent();
> >         checkIntent.setAction
> > (TextToSpeech.Engine.ACTION_CHECK_TTS_DATA);
> >         startActivityForResult(checkIntent, REQ_TTS_STATUS_CHECK);
> >     }
>
> >     public void startStop(View view) {
> >        if(mTts.isSpeaking()) {
> >                 startStopBtn.setText("Start");
> >                 mTts.stop();
> >         }
> >         else {
> >                 mTts.setOnUtteranceCompletedListener(this);
> >                 startStopBtn.setText("Stop");
> >                 sayNext();
> >         }
> >         };
>
> >         private void sayNext() {
> >                 params.put(TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID,
> >                 "value");
> >         mTts.speak(String.valueOf(counter++), TextToSpeech.QUEUE_ADD,
> > params);
> >                 params.put(TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID,
> >             "silence");
> >         mTts.playSilence(2000, TextToSpeech.QUEUE_ADD, params);
> >         }
>
> >     protected void onActivityResult(int requestCode, int resultCode,
> > Intent data) {

> >        if(requestCode == REQ_TTS_STATUS_CHECK) {

> >             switch (resultCode) {
> >             case TextToSpeech.Engine.CHECK_VOICE_DATA_PASS:
> >                 // TTS is up and running
> >                 mTts = new TextToSpeech(this, this);
> >                 Log.v(TAG, "Pico is installed okay");
> >                 break;
> >             case TextToSpeech.Engine.CHECK_VOICE_DATA_BAD_DATA:
> >             case TextToSpeech.Engine.CHECK_VOICE_DATA_MISSING_DATA:
> >             case TextToSpeech.Engine.CHECK_VOICE_DATA_MISSING_VOLUME:
> >                 // missing data, install it

> >                 Log.v(TAG, "Needlanguagestuff: " + resultCode);

> >                 Intent installIntent = new Intent();
> >                 installIntent.setAction(
> >                     TextToSpeech.Engine.ACTION_INSTALL_TTS_DATA);
> >                 startActivity(installIntent);
> >                 break;
> >             case TextToSpeech.Engine.CHECK_VOICE_DATA_FAIL:
> >             default:
> >                 Log.e(TAG, "Got a failure. TTS apparently not
> > available");
> >             }
> >         }
> >         else {
> >                 // Got something else
> >         }
> >     }
>
> >         @Override
> >         public void onInit(int status) {
> >                 // Now that the TTS engine is ready, we enable the button
> >                if( status == TextToSpeech.SUCCESS) {
> >                         startStopBtn.setEnabled(true);
> >                 }
> >         }
>
> >         @Override
> >         public void onPause()
> >         {
> >                 super.onPause();
> >               Log.v(TAG,"In onPause()");
> >         }
>
> >         @Override
> >     public void onDestroy()
> >         {
> >         super.onDestroy();

> >                 //ifwe're going away, clean up

> >                if( mTts != null) {
> >                         mTts.stop();
> >             mTts.shutdown();
> >                 }
> >     }
>
> >         @Override
> >         public void onUtteranceCompleted(String uttId) {
> >                 Log.v(TAG,"Got utterance: " + uttId);
> >                if("silence".compareTo(uttId) == 0)
> >                     sayNext();
> >         }
>
> > }
>
> > Run it and click the Start button. Now hit the Home button and start
> > up some other application. You should continue to hear numbers being
> > counted to you even though you're now using another application.
> > Pretty soon I think you'll realize why applications "go to sleep" when

> > they lose focus.Ifyou change the duration argument of playSilence to

> > a smaller number than 2000 (100 did it for me), you will find that
> > Android decides to kill the application a little while after it has
> > been backgrounded.
>
> > - dave
>
> > On Oct 29, 4:48 pm, blindfold <seeingwithso...@gmail.com> wrote:
>
> > > Thank you for your answers, Charles! I will consider posting to
> > > android-developers, but had posted here because I expect that Google
> > > Maps Navigation uses the same TTS functionnality for its spoken
> > > directions that is the focus of this group in discussing interactions
> > > with third-party self-voicing applications and system-wide screen
> > > readers. In short, I'd think the most knowledgeable people on this
> > > theme are here.
>
> > > > Android does run multiple applications simultaneously; in fact,
> > > > having the ability to run an app in the background is one of the
> > > > Android platform's features.
>
> > > It depends on what one means by "running", but my understanding is
> > > that a normal Android app, by which I here mean an Activity, does not

> > > perform *any* taskifnot in the foreground, and can be completely

> > > killed by the system at any moment, e.g. when short on resources. It
> > > is essentially in hibernation until brought back to the foreground.
> > > Alive yes, with its state preserved, but not "running" in a functional
> > > sense. That is what I meant by saying that Android runs only one app

> > > at a time, because this would imply thatifGoogle Maps Navigation is

> > > an Activity it would not be giving directions while not in the
> > > foreground, that is, when my own app (an Activity) is running in the
> > > foreground. *Only* when Google Maps Navigation is a *Service* could it
> > > (but *does* it also?) continue to give spoken directions while my app
> > > is running in the foreground.
>

> > > Now I do not currently knowifGoogle Maps Navigation is an Activity

> > > or a Service, hence my inquiry.
>
> > > > As far as the Maps APIs go, please see the API reference here:
> > > >http://code.google.com/android/add-ons/google-apis/reference/index.html
>
> > >http://code.google.com/android/add-ons/google-apis/maps-api-signup.html
>
> > > reads "The Android Maps APIs explicitly do not include any driving
> > > directions data or local search data that may be owned or licensed by
> > > Google." So Google Maps Navigation goes beyond the Maps APIs by giving

> > > actual driving directions, hence the questionifone can tap into that

> > > newly added functionality through some other or extended API, or
> > > whether one can at best hope to have Google Maps Navigation running -
> > > and still speaking - as a Service while one's own app Activity is
> > > running in the foreground.
>
> > > > P.S. Peter, in your specific case, have you considered using Google
> > > > walking directions instead?
>
> > > Yes, that will be a useful refinement. I had assumed that one could
>

> ...
>
> read more »