[mailto:cm-aac-forum@googlegroups.com] On Behalf Of Christopher Amery Sent: 11 April 2012 11:14 To: CM AAC Forum Subject: [AAC Forum] Student Needing Further Help
Hello all,
As some of you may remember, I tried to compile a survey in relation to Jolly-Phonic hand shapes within the context of visual recognition a while back, which prompted further investigation. After some further research upon a reportedly popular system known as Cued Speech, i've decided to compile yet another very short survey to gather feedback and opinion in relation to using Cued Speech under visual recognition, as there is very little to report within this area. If any of you have a spare moment to help me with my research, that would be most appreciated!
Any questions, please don't hesitate to contact me!
Thanks, Chris
-- GUIDELINES - COMMUNICATION MATTERS AAC FORUM
1. The purpose of this email group is for people to share experiences and information about Augmentative and Alternative Communication (AAC). The views and opinions expressed are those of individuals and do not represent the views of Communication Matters.
2. This email group must not be used for advertising or promoting goods or services.
3. * COMMERCIAL INTERESTS * If you are a supplier or have commercial interests, you must preface each of your posts with the words 'COMMERCIAL INTERESTS DECLARED'.
Although this group is not actively moderated, Communication Matters reserves the right to remove messages and members considered offensive, libellous, or not following the above guidelines.
To post to this group, email: cm-aac-forum@googlegroups.com To unsubscribe from this group, email: cm-aac-forum+unsubscribe@googlegroups.com For more options, visit: http://groups.google.com/group/cm-aac-forum
This message is intended for the use of the addressee only and may contain confidential or privileged information. If you have received it in error please notify the sender and destroy it. You may not use it or copy it to anyone else.
E-mail is not a secure communications medium. Please be aware of this when replying. All communications sent to or from the County Council may be subject to recording and/or monitoring in accordance with relevant legislation.
Although East Sussex County Council has taken steps to ensure that this e-mail and any attachments are virus free, we can take no responsibility if a virus is actually present and you are advised to ensure that the appropriate checks are made.
I just don;t quite understand how a gesture recognition system would help an AAC user....are you looking at whether Jolly phonics can be used to basically "spell" out words? Maybe it is me but i just dont quite follow at all. Sorry
Dear Chris I have looked at your links and I cannot quite understand what you are trying to achieve either. I think I got the same impression as Jenny. Are you trying to see if an AAC device could recognise a hand gesture and translate this into a spoken form? Is this intended to help an AAC user with their literacy development by using their communication aid to provide a spoken sound for a gesture made by another person? Sorry I have got in a muddle with this too! Amanda
________________________________________ From: cm-aac-forum@googlegroups.com [cm-aac-forum@googlegroups.com] on behalf of Jenny Herd [je...@herdy.org.uk] Sent: 12 April 2012 23:27 To: cm-aac-forum@googlegroups.com Subject: Re: [AAC Forum] Re: Student Needing Further Help
I just don;t quite understand how a gesture recognition system would help an AAC user....are you looking at whether Jolly phonics can be used to basically "spell" out words? Maybe it is me but i just dont quite follow at all. Sorry
> If anyone could respond to this very short survey for my dissertation/ > thesis, it would be highly appreciated!
> Thanks, Chris
-- GUIDELINES - COMMUNICATION MATTERS AAC FORUM
1. The purpose of this email group is for people to share experiences and information about Augmentative and Alternative Communication (AAC). The views and opinions expressed are those of individuals and do not represent the views of Communication Matters.
2. This email group must not be used for advertising or promoting goods or services.
3. * COMMERCIAL INTERESTS * If you are a supplier or have commercial interests, you must preface each of your posts with the words 'COMMERCIAL INTERESTS DECLARED'.
Although this group is not actively moderated, Communication Matters reserves the right to remove messages and members considered offensive, libellous, or not following the above guidelines.
To post to this group, email: cm-aac-forum@googlegroups.com To unsubscribe from this group, email: cm-aac-forum+unsubscribe@googlegroups.com For more options, visit: http://groups.google.com/group/cm-aac-forum
I have had a go at completing the survey and had a look some of the previous messages from when you first posted here in February.
I have not "returned" the survey because you might still be revising it and I thought it might help to post comments here rather than in the "Comments" section of the survey itself. (I am assuming that once a survey form has been completed and returned then it would not be possible to complete it a second time.)
*Q1. **How important do you value the following functionality or features for a technological AAC intervention involved in recognising visual gestures?*
*Comment:* My replies to this question would be based on imagining that the device would recognise the visual gestures made by the AAC-user, in order to assist the communication partner to understand the message. This would be similar in concept to an device that "decoded", for example, BSL or Makaton or Fingerspelling production for the benefit of a communication partner who did not understand these systems. Am I right in thinking that this is how you envisage the device being used? Or, if not, that this is still a way in which the device could be used?
*Q2. How much value do you place on the ability to utilise the following phonological systems for an AAC intervention specialising in visual recognition? (This question aims to evaluate the potential suitability for alternative systems to be utilised to facilitate gestures.)*
*Comment:* The systems listed are Jolly-Phonics, Cued Articulation and Cued Speech. I thought I understood this question correctly at first but then I noticed the explanation, "*This question aims to evaluate the potential suitability for alternative systems to be utilised to facilitate gestures"*and now I am not so sure. What I thought you had in mind was a device that aimed to facilitate communication using gestures, by "decoding" the gestures, rather than a device that facilitated the gestures themselves. Did you mean to say that the device would aim to *"facilitate communication using gestures"*?
The options for Q1 include personalisation of the device but for Q2 are restricted to the three named systems. The following comment is based on my expectation, perhaps wrong, that you are considering a device that might *"facilitate communication using gestures":*
I appreciate that your starting point was Jolly-Phonics but, having widened the questions to include Cued Speech and Cued Articulation, I wonder if it might be helpful to leave space for respondents to add other systems that might be used in a similar way, ie. production of other hand shapes close to the face while speaking? I say this because I am aware that one-handed fingerspelling is sometimes used in this way, although it is not a *"phonological system"* of course.
*Q3. Evaluate upon the potential suitability of visual gesture based interaction as a method to facilitate communication upon the following groups of people.* *(This question aims to investigate the suited target audience for visual gesture recognition AAC systems.)*
*Comment: **"to facilitate communication upon the following groups of people"*: *"with"* or *"by"* rather than *"upon"*? * *The rating scale for the examples of *"people who experience . . . "*(list of descriptions) is: Highly Suitable Quite Suitable Neither Suitable or Useless Quite Useless Highly Useless
I would really like an option to say that I cannot comment on an example description rather than have to choose "*Neither Suitable or Useless*". (I assume this option should read *"Neither Suitable NOR Useless"*.)
Perhaps I am having difficulty with some aspects of the the survey because my experience of using visual gesture with AAC users is rather limited and primarily with people who acquired a severe speech problem in adulthood. I used a "pick and mix" approach: first, identifying the minimum number of gestures needed to help clarify ambiguous speech sounds; next, narrowing down further based on preference and ease of production for the individual. For example, in the few instances where I have used this approach, I found that the sign for "m" in one-handed fingerspelling was preferred over the equivalent signs in Cued Speech and Cued Articulation. This was because: the handshape looks more like the letter "m" than the other two and learning and reinforced using drawings to emphasise the similarity between the handshape, the letter "m" and the shape of the upper lip.
I hope these comments are helpful, * Liz Panton MRCSLT*
> I just don;t quite understand how a gesture recognition system would help > an AAC user....are you looking at whether Jolly phonics can be used to > basically "spell" out words? Maybe it is me but i just dont quite follow > at all. > Sorry
>> If anyone could respond to this very short survey for my dissertation/ >> thesis, it would be highly appreciated!
>> Thanks, Chris
> -- > GUIDELINES - COMMUNICATION MATTERS AAC FORUM > 1. The purpose of this email group is for people to share experiences and > information about Augmentative and Alternative Communication (AAC). The > views and opinions expressed are those of individuals and do not represent > the views of Communication Matters. > 2. This email group must not be used for advertising or promoting goods or > services. > 3. * COMMERCIAL INTERESTS * > If you are a supplier or have commercial interests, you must preface each > of your posts with the words 'COMMERCIAL INTERESTS DECLARED'.
> Although this group is not actively moderated, Communication Matters > reserves the right to remove messages and members considered offensive, > libellous, or not following the above guidelines.
It's a piece of software utilising a camera that's intended to help an
AAC user (a target audience of which i'm trying to figure out), by
enabling them to present gestures in front of the camera which
represent different phonemes within the Jolly-Phonics programme. The
software would recognise these gestures presented in front of a camera
and construct simple words depending on the phoneme gestures
presented.
Because of previous responses i've had, it should be emphasised that
the gestures used aren't necessarily representative of 'Finger
Phonics' used within the Jolly-Phonics programme. If you've ever heard
of the 'Phonic Stick' project that was conducted at Dundee University
a few years ago, my aim is somewhat similar to the scope of that
project, only visual recognition with standard webcams is used instead
of a stick.
If you wish me to illustrate what i'm doing more simply with a diagram
of such, then i'll be happy to do so if it helps to clear any
confusion!
<AMANDA.E.HY...@stu.mmu.ac.uk> wrote:
> Dear Chris
> I have looked at your links and I cannot quite understand what you are trying to achieve either. I think I got the same impression as Jenny.
> Are you trying to see if an AAC device could recognise a hand gesture and translate this into a spoken form? Is this intended to help an AAC user with their literacy development by using their communication aid to provide a spoken sound for a gesture made by another person?
> Sorry I have got in a muddle with this too!
> Amanda
> ________________________________________
> From: cm-aac-forum@googlegroups.com [cm-aac-forum@googlegroups.com] on behalf of Jenny Herd [je...@herdy.org.uk]
> Sent: 12 April 2012 23:27
> To: cm-aac-forum@googlegroups.com
> Subject: Re: [AAC Forum] Re: Student Needing Further Help
> I just don;t quite understand how a gesture recognition system would
> help an AAC user....are you looking at whether Jolly phonics can be used
> to basically "spell" out words? Maybe it is me but i just dont quite
> follow at all.
> Sorry
> > If anyone could respond to this very short survey for my dissertation/
> > thesis, it would be highly appreciated!
> > Thanks, Chris
> --
> GUIDELINES - COMMUNICATION MATTERS AAC FORUM
> 1. The purpose of this email group is for people to share experiences and information about Augmentative and Alternative Communication (AAC). The views and opinions expressed are those of individuals and do not represent the views of Communication Matters.
> 2. This email group must not be used for advertising or promoting goods or services.
> 3. * COMMERCIAL INTERESTS *
> If you are a supplier or have commercial interests, you must preface each of your posts with the words 'COMMERCIAL INTERESTS DECLARED'.
> Although this group is not actively moderated, Communication Matters reserves the right to remove messages and members considered offensive, libellous, or not following the above guidelines.
> To post to this group, email: cm-aac-forum@googlegroups.com
> To unsubscribe from this group, email: cm-aac-forum+unsubscribe@googlegroups.com
> For more options, visit:http://groups.google.com/group/cm-aac-forum
Hey Liz, thanks for your comprehensive post. You've brought up a
couple of good points.
In your response to question 1, that's exactly how I picture it. This
is a system that would 'decode' gestures that represent Jolly-Phonics
and allow an AAC user to blend and construct different phonemes
together in order to make a word. The system would produce this word
in the output form such as synthetic speech.
In your response to question 2, there is potential for vision based
recognition systems to be 'diversified' for other uses. While my
primary aim is to construct a prototype system which recognises Jolly-
Phonic phonemes and blend these together to output words, the
technical and mechanical workings of the software for the ability to
recognise and associate gestures could be adapted to recognise other
gestures, such as Cued Speech or Cued Articulation gestures. This
question simply aims to evaluate whether these alternative systems
such as Cued Speech or Cued Articulation would benefit more from a
visual recognition system.
In response to question 3, this is a limitation of Google Documents,
where only five choices can be added. I'll try and amend it
accordingly though.
However, your suggestion to include a comment box for alternative
systems which i've not picked up on is something i'll include
momentarily.
Thanks, Chris
On Apr 13, 12:23 pm, "Liz Panton <l...@panton.me.uk>"
> I have had a go at completing the survey and had a look some of the
> previous messages from when you first posted here in February.
> I have not "returned" the survey because you might still be revising it and
> I thought it might help to post comments here rather than in the "Comments"
> section of the survey itself. (I am assuming that once a survey form has
> been completed and returned then it would not be possible to complete it a
> second time.)
> *Q1. **How important do you value the following functionality or features
> for a technological AAC intervention involved in recognising visual
> gestures?*
> *Comment:* My replies to this question would be based on imagining that the
> device would recognise the visual gestures made by the AAC-user, in order
> to assist the communication partner to understand the message. This would
> be similar in concept to an device that "decoded", for example, BSL or
> Makaton or Fingerspelling production for the benefit of a communication
> partner who did not understand these systems. Am I right in thinking that
> this is how you envisage the device being used? Or, if not, that this is
> still a way in which the device could be used?
> *Q2. How much value do you place on the ability to utilise the following
> phonological systems for an AAC intervention specialising in visual
> recognition? (This question aims to evaluate the potential suitability for
> alternative systems to be utilised to facilitate gestures.)*
> *Comment:* The systems listed are Jolly-Phonics, Cued Articulation and Cued
> Speech. I thought I understood this question correctly at first but then I
> noticed the explanation, "*This question aims to evaluate the potential
> suitability for alternative systems to be utilised to facilitate
> gestures"*and now I am not so sure. What I thought you had in mind
> was a device that
> aimed to facilitate communication using gestures, by "decoding" the
> gestures, rather than a device that facilitated the gestures themselves.
> Did you mean to say that the device would aim to *"facilitate communication
> using gestures"*?
> The options for Q1 include personalisation of the device but for Q2 are
> restricted to the three named systems. The following comment is based on
> my expectation, perhaps wrong, that you are considering a device that might
> *"facilitate communication using gestures":*
> I appreciate that your starting point was Jolly-Phonics but, having widened
> the questions to include Cued Speech and Cued Articulation, I wonder if it
> might be helpful to leave space for respondents to add other systems that
> might be used in a similar way, ie. production of other hand shapes close
> to the face while speaking? I say this because I am aware that one-handed
> fingerspelling is sometimes used in this way, although it is not a
> *"phonological
> system"* of course.
> *Q3. Evaluate upon the potential suitability of visual gesture based
> interaction as a method to facilitate communication upon the following
> groups of people.* *(This question aims to investigate the suited target
> audience for visual gesture recognition AAC systems.)*
> *Comment: **"to facilitate communication upon the following groups of
> people"*: *"with"* or *"by"* rather than *"upon"*?
> *
> *The rating scale for the examples of *"people who experience . . .
> "*(list of descriptions) is:
> Highly Suitable Quite Suitable Neither Suitable or Useless Quite
> Useless Highly
> Useless
> I would really like an option to say that I cannot comment on an example
> description rather than have to choose "*Neither Suitable or Useless*". (I
> assume this option should read *"Neither Suitable NOR Useless"*.)
> Perhaps I am having difficulty with some aspects of the the survey because
> my experience of using visual gesture with AAC users is rather limited and
> primarily with people who acquired a severe speech problem in adulthood. I
> used a "pick and mix" approach: first, identifying the minimum number of
> gestures needed to help clarify ambiguous speech sounds; next, narrowing
> down further based on preference and ease of production for the
> individual. For example, in the few instances where I have used this
> approach, I found that the sign for "m" in one-handed fingerspelling was
> preferred over the equivalent signs in Cued Speech and Cued Articulation.
> This was because: the handshape looks more like the letter "m" than the
> other two and learning and reinforced using drawings to emphasise the
> similarity between the handshape, the letter "m" and the shape of the upper
> lip.
> I hope these comments are helpful,
> *
> Liz Panton MRCSLT*
> On 12 April 2012 23:27, Jenny Herd <je...@herdy.org.uk> wrote:
> > I just don;t quite understand how a gesture recognition system would help
> > an AAC user....are you looking at whether Jolly phonics can be used to
> > basically "spell" out words? Maybe it is me but i just dont quite follow
> > at all.
> > Sorry
> >> If anyone could respond to this very short survey for my dissertation/
> >> thesis, it would be highly appreciated!
> >> Thanks, Chris
> > --
> > GUIDELINES - COMMUNICATION MATTERS AAC FORUM
> > 1. The purpose of this email group is for people to share experiences and
> > information about Augmentative and Alternative Communication (AAC). The
> > views and opinions expressed are those of individuals and do not represent
> > the views of Communication Matters.
> > 2. This email group must not be used for advertising or promoting goods or
> > services.
> > 3. * COMMERCIAL INTERESTS *
> > If you are a supplier or have commercial interests, you must preface each
> > of your posts with the words 'COMMERCIAL INTERESTS DECLARED'.
> > Although this group is not actively moderated, Communication Matters
> > reserves the right to remove messages and members considered offensive,
> > libellous, or not following the above guidelines.
Hi Chris Many AAC users are likely to have difficulty with motor movement (hence the reason their vocal muscles are affected) and I do not know how easy it would be for them to produce visual signs. However, there are many client groups for whom AAC might be applicable. I wonder if you are thinking of people on the autistic spectrum or those with learning difficulties who may not have the associated motor difficulties? Many people who use AAC are likely to have conditions such as cerebral palsy, suffered a stroke, motor neuron disease, amyotrophic lateral sclerosis and Parkinsons. For many of these clients one of the main challenges of intervention is helping them to access a communication aid due to their motor co-ordination difficulties. Sometimes they require keyguards or an indirect form of access for example through switch technology (e.g. a joystick or head switch) or eyegaze. I am aware of the PhonicStick project at the University of Dundee. I believe it lets users operate a joystick to access sounds. I presume as you mention this maybe you are not expecting people to make specific hand shapes. Are you planning to somehow build software that would recognise their own individual gestures and so build a kind of personalised recognition system tailored to an individual's particular skills? Anyway, as I said I may have got into a muddle with what you are trying to do. Amanda
________________________________________ From: cm-aac-forum@googlegroups.com [cm-aac-forum@googlegroups.com] on behalf of Christopher Amery [amerychristop...@gmail.com] Sent: 13 April 2012 12:24 To: CM AAC Forum Subject: [AAC Forum] Re: Student Needing Further Help
Hey Amanda,
It's a piece of software utilising a camera that's intended to help an AAC user (a target audience of which i'm trying to figure out), by enabling them to present gestures in front of the camera which represent different phonemes within the Jolly-Phonics programme. The software would recognise these gestures presented in front of a camera and construct simple words depending on the phoneme gestures presented.
Because of previous responses i've had, it should be emphasised that the gestures used aren't necessarily representative of 'Finger Phonics' used within the Jolly-Phonics programme. If you've ever heard of the 'Phonic Stick' project that was conducted at Dundee University a few years ago, my aim is somewhat similar to the scope of that project, only visual recognition with standard webcams is used instead of a stick.
If you wish me to illustrate what i'm doing more simply with a diagram of such, then i'll be happy to do so if it helps to clear any confusion!
<AMANDA.E.HY...@stu.mmu.ac.uk> wrote: > Dear Chris > I have looked at your links and I cannot quite understand what you are trying to achieve either. I think I got the same impression as Jenny. > Are you trying to see if an AAC device could recognise a hand gesture and translate this into a spoken form? Is this intended to help an AAC user with their literacy development by using their communication aid to provide a spoken sound for a gesture made by another person? > Sorry I have got in a muddle with this too! > Amanda
> ________________________________________ > From: cm-aac-forum@googlegroups.com [cm-aac-forum@googlegroups.com] on behalf of Jenny Herd [je...@herdy.org.uk] > Sent: 12 April 2012 23:27 > To: cm-aac-forum@googlegroups.com > Subject: Re: [AAC Forum] Re: Student Needing Further Help
> I just don;t quite understand how a gesture recognition system would > help an AAC user....are you looking at whether Jolly phonics can be used > to basically "spell" out words? Maybe it is me but i just dont quite > follow at all. > Sorry
> > If anyone could respond to this very short survey for my dissertation/ > > thesis, it would be highly appreciated!
> > Thanks, Chris
> -- > GUIDELINES - COMMUNICATION MATTERS AAC FORUM
> 1. The purpose of this email group is for people to share experiences and information about Augmentative and Alternative Communication (AAC). The views and opinions expressed are those of individuals and do not represent the views of Communication Matters.
> 2. This email group must not be used for advertising or promoting goods or services.
> 3. * COMMERCIAL INTERESTS * > If you are a supplier or have commercial interests, you must preface each of your posts with the words 'COMMERCIAL INTERESTS DECLARED'.
> Although this group is not actively moderated, Communication Matters reserves the right to remove messages and members considered offensive, libellous, or not following the above guidelines.
> To post to this group, email: cm-aac-forum@googlegroups.com > To unsubscribe from this group, email: cm-aac-forum+unsubscribe@googlegroups.com > For more options, visit:http://groups.google.com/group/cm-aac-forum
-- GUIDELINES - COMMUNICATION MATTERS AAC FORUM
1. The purpose of this email group is for people to share experiences and information about Augmentative and Alternative Communication (AAC). The views and opinions expressed are those of individuals and do not represent the views of Communication Matters.
2. This email group must not be used for advertising or promoting goods or services.
3. * COMMERCIAL INTERESTS * If you are a supplier or have commercial interests, you must preface each of your posts with the words 'COMMERCIAL INTERESTS DECLARED'.
Although this group is not actively moderated, Communication Matters reserves the right to remove messages and members considered offensive, libellous, or not following the above guidelines.
To post to this group, email: cm-aac-forum@googlegroups.com To unsubscribe from this group, email: cm-aac-forum+unsubscribe@googlegroups.com For more options, visit: http://groups.google.com/group/cm-aac-forum
This may be a bit of tangent for your research, but a team at City University has been looking at how to use computerised training programmes to teach gesture as a means of communication. They are working with adults who have aphasia, usually as a result of stroke. The computer software they have developed teaches specific gestures (eg 'umbrella' or 'glasses') and also provides virtual environments where people can respond to what is happening via gesture.
Reg. charity Number: 1061636 FAST is a limited company no. 3331184 (England and Wales)
Registered Address: FAST, c/o King Loose & Co, St John's House, 5 South Parade, Summertown, Oxford OX2 7JL
To stay up to date with the latest news on assistive technology why not subscribe to FAST's free monthly e-newsletters. Send an email to i...@fastuk.org to subscribe to AT Policy News. To subscribe to AT Research and Development News send an email to resea...@fastuk.org.
-----Original Message----- From: cm-aac-forum@googlegroups.com [mailto:cm-aac-forum@googlegroups.com] On Behalf Of Christopher Amery Sent: 13 April 2012 12:39 To: CM AAC Forum Subject: [AAC Forum] Re: Student Needing Further Help
Hey Liz, thanks for your comprehensive post. You've brought up a couple of good points.
In your response to question 1, that's exactly how I picture it. This is a system that would 'decode' gestures that represent Jolly-Phonics and allow an AAC user to blend and construct different phonemes together in order to make a word. The system would produce this word in the output form such as synthetic speech.
In your response to question 2, there is potential for vision based recognition systems to be 'diversified' for other uses. While my primary aim is to construct a prototype system which recognises Jolly- Phonic phonemes and blend these together to output words, the technical and mechanical workings of the software for the ability to recognise and associate gestures could be adapted to recognise other gestures, such as Cued Speech or Cued Articulation gestures. This question simply aims to evaluate whether these alternative systems such as Cued Speech or Cued Articulation would benefit more from a visual recognition system.
In response to question 3, this is a limitation of Google Documents, where only five choices can be added. I'll try and amend it accordingly though.
However, your suggestion to include a comment box for alternative systems which i've not picked up on is something i'll include momentarily.
Thanks, Chris
On Apr 13, 12:23 pm, "Liz Panton <l...@panton.me.uk>" <l...@panton.me.uk> wrote: > Hi Christopher,
> I have had a go at completing the survey and had a look some of the > previous messages from when you first posted here in February.
> I have not "returned" the survey because you might still be revising it and > I thought it might help to post comments here rather than in the "Comments" > section of the survey itself. (I am assuming that once a survey form has > been completed and returned then it would not be possible to complete it a > second time.)
> *Q1. **How important do you value the following functionality or features > for a technological AAC intervention involved in recognising visual > gestures?*
> *Comment:* My replies to this question would be based on imagining that the > device would recognise the visual gestures made by the AAC-user, in order > to assist the communication partner to understand the message. This would > be similar in concept to an device that "decoded", for example, BSL or > Makaton or Fingerspelling production for the benefit of a communication > partner who did not understand these systems. Am I right in thinking that > this is how you envisage the device being used? Or, if not, that this is > still a way in which the device could be used?
> *Q2. How much value do you place on the ability to utilise the following > phonological systems for an AAC intervention specialising in visual > recognition? (This question aims to evaluate the potential suitability for > alternative systems to be utilised to facilitate gestures.)*
> *Comment:* The systems listed are Jolly-Phonics, Cued Articulation and Cued > Speech. I thought I understood this question correctly at first but then I > noticed the explanation, "*This question aims to evaluate the potential > suitability for alternative systems to be utilised to facilitate > gestures"*and now I am not so sure. What I thought you had in mind > was a device that > aimed to facilitate communication using gestures, by "decoding" the > gestures, rather than a device that facilitated the gestures themselves. > Did you mean to say that the device would aim to *"facilitate communication > using gestures"*?
> The options for Q1 include personalisation of the device but for Q2 are > restricted to the three named systems. The following comment is based on > my expectation, perhaps wrong, that you are considering a device that might > *"facilitate communication using gestures":*
> I appreciate that your starting point was Jolly-Phonics but, having widened > the questions to include Cued Speech and Cued Articulation, I wonder if it > might be helpful to leave space for respondents to add other systems that > might be used in a similar way, ie. production of other hand shapes close > to the face while speaking? I say this because I am aware that one-handed > fingerspelling is sometimes used in this way, although it is not a > *"phonological > system"* of course.
> *Q3. Evaluate upon the potential suitability of visual gesture based > interaction as a method to facilitate communication upon the following > groups of people.* *(This question aims to investigate the suited target > audience for visual gesture recognition AAC systems.)*
> *Comment: **"to facilitate communication upon the following groups of > people"*: *"with"* or *"by"* rather than *"upon"*? > * > *The rating scale for the examples of *"people who experience . . . > "*(list of descriptions) is: > Highly Suitable Quite Suitable Neither Suitable or Useless Quite > Useless Highly > Useless
> I would really like an option to say that I cannot comment on an example > description rather than have to choose "*Neither Suitable or Useless*". (I > assume this option should read *"Neither Suitable NOR Useless"*.)
> Perhaps I am having difficulty with some aspects of the the survey because > my experience of using visual gesture with AAC users is rather limited and > primarily with people who acquired a severe speech problem in adulthood. I > used a "pick and mix" approach: first, identifying the minimum number of > gestures needed to help clarify ambiguous speech sounds; next, narrowing > down further based on preference and ease of production for the > individual. For example, in the few instances where I have used this > approach, I found that the sign for "m" in one-handed fingerspelling was > preferred over the equivalent signs in Cued Speech and Cued Articulation. > This was because: the handshape looks more like the letter "m" than the > other two and learning and reinforced using drawings to emphasise the > similarity between the handshape, the letter "m" and the shape of the upper > lip.
> I hope these comments are helpful, > * > Liz Panton MRCSLT*
> On 12 April 2012 23:27, Jenny Herd <je...@herdy.org.uk> wrote:
> > I just don;t quite understand how a gesture recognition system would help > > an AAC user....are you looking at whether Jolly phonics can be used to > > basically "spell" out words? Maybe it is me but i just dont quite follow > > at all. > > Sorry
> >> If anyone could respond to this very short survey for my dissertation/ > >> thesis, it would be highly appreciated!
> >> Thanks, Chris
> > -- > > GUIDELINES - COMMUNICATION MATTERS AAC FORUM > > 1. The purpose of this email group is for people to share experiences and > > information about Augmentative and Alternative Communication (AAC). The > > views and opinions expressed are those of individuals and do not represent > > the views of Communication Matters. > > 2. This email group must not be used for advertising or promoting goods or > > services. > > 3. * COMMERCIAL INTERESTS * > > If you are a supplier or have commercial interests, you must preface each > > of your posts with the words 'COMMERCIAL INTERESTS DECLARED'.
> > Although this group is not actively moderated, Communication Matters > > reserves the right to remove messages and members considered offensive, > > libellous, or not following the above guidelines.
> > To post to this group, email: cm-aac-forum@googlegroups.com > > To unsubscribe from this group, email: cm-aac-forum+unsubscribe@** > > googlegroups.com
I may ask for further details later next week, but thanks for letting
me know about this!
In relation to the survey which is now final (i.e. I won't be changing
it from here on out), i've had a couple of responses so far, which is
much appreciated!
In response to the last question on your post Amanda, the software
could be made to work that way. I'm simply trying to elicit important
requirements for a gesture recognition tool for and beyond and (maybe)
beyond prototype stages.
Thanks, Chris
On Apr 13, 1:56 pm, Pat Sweet <P...@fastuk.org> wrote:
> This may be a bit of tangent for your research, but a team at City University has been looking at how to use computerised training programmes to teach gesture as a means of communication. They are working with adults who have aphasia, usually as a result of stroke. The computer software they have developed teaches specific gestures (eg 'umbrella' or 'glasses') and also provides virtual environments where people can respond to what is happening via gesture.
> Reg. charity Number: 1061636
> FAST is a limited company no. 3331184 (England and Wales)
> Registered Address: FAST, c/o King Loose & Co, St John's House, 5 South Parade, Summertown, Oxford OX2 7JL
> To stay up to date with the latest news on assistive technology why not subscribe to FAST's free monthly e-newsletters. Send an email to i...@fastuk.org to subscribe to AT Policy News. To subscribe to AT Research and Development News send an email to resea...@fastuk.org.
> fastsmall
> -----Original Message-----
> From: cm-aac-forum@googlegroups.com [mailto:cm-aac-forum@googlegroups.com] On Behalf Of Christopher Amery
> Sent: 13 April 2012 12:39
> To: CM AAC Forum
> Subject: [AAC Forum] Re: Student Needing Further Help
> Hey Liz, thanks for your comprehensive post. You've brought up a
> couple of good points.
> In your response to question 1, that's exactly how I picture it. This
> is a system that would 'decode' gestures that represent Jolly-Phonics
> and allow an AAC user to blend and construct different phonemes
> together in order to make a word. The system would produce this word
> in the output form such as synthetic speech.
> In your response to question 2, there is potential for vision based
> recognition systems to be 'diversified' for other uses. While my
> primary aim is to construct a prototype system which recognises Jolly-
> Phonic phonemes and blend these together to output words, the
> technical and mechanical workings of the software for the ability to
> recognise and associate gestures could be adapted to recognise other
> gestures, such as Cued Speech or Cued Articulation gestures. This
> question simply aims to evaluate whether these alternative systems
> such as Cued Speech or Cued Articulation would benefit more from a
> visual recognition system.
> In response to question 3, this is a limitation of Google Documents,
> where only five choices can be added. I'll try and amend it
> accordingly though.
> However, your suggestion to include a comment box for alternative
> systems which i've not picked up on is something i'll include
> momentarily.
> Thanks, Chris
> On Apr 13, 12:23 pm, "Liz Panton <l...@panton.me.uk>"
> <l...@panton.me.uk> wrote:
> > Hi Christopher,
> > I have had a go at completing the survey and had a look some of the
> > previous messages from when you first posted here in February.
> > I have not "returned" the survey because you might still be revising it and
> > I thought it might help to post comments here rather than in the "Comments"
> > section of the survey itself. (I am assuming that once a survey form has
> > been completed and returned then it would not be possible to complete it a
> > second time.)
> > *Q1. **How important do you value the following functionality or features
> > for a technological AAC intervention involved in recognising visual
> > gestures?*
> > *Comment:* My replies to this question would be based on imagining that the
> > device would recognise the visual gestures made by the AAC-user, in order
> > to assist the communication partner to understand the message. This would
> > be similar in concept to an device that "decoded", for example, BSL or
> > Makaton or Fingerspelling production for the benefit of a communication
> > partner who did not understand these systems. Am I right in thinking that
> > this is how you envisage the device being used? Or, if not, that this is
> > still a way in which the device could be used?
> > *Q2. How much value do you place on the ability to utilise the following
> > phonological systems for an AAC intervention specialising in visual
> > recognition? (This question aims to evaluate the potential suitability for
> > alternative systems to be utilised to facilitate gestures.)*
> > *Comment:* The systems listed are Jolly-Phonics, Cued Articulation and Cued
> > Speech. I thought I understood this question correctly at first but then I
> > noticed the explanation, "*This question aims to evaluate the potential
> > suitability for alternative systems to be utilised to facilitate
> > gestures"*and now I am not so sure. What I thought you had in mind
> > was a device that
> > aimed to facilitate communication using gestures, by "decoding" the
> > gestures, rather than a device that facilitated the gestures themselves.
> > Did you mean to say that the device would aim to *"facilitate communication
> > using gestures"*?
> > The options for Q1 include personalisation of the device but for Q2 are
> > restricted to the three named systems. The following comment is based on
> > my expectation, perhaps wrong, that you are considering a device that might
> > *"facilitate communication using gestures":*
> > I appreciate that your starting point was Jolly-Phonics but, having widened
> > the questions to include Cued Speech and Cued Articulation, I wonder if it
> > might be helpful to leave space for respondents to add other systems that
> > might be used in a similar way, ie. production of other hand shapes close
> > to the face while speaking? I say this because I am aware that one-handed
> > fingerspelling is sometimes used in this way, although it is not a
> > *"phonological
> > system"* of course.
> > *Q3. Evaluate upon the potential suitability of visual gesture based
> > interaction as a method to facilitate communication upon the following
> > groups of people.* *(This question aims to investigate the suited target
> > audience for visual gesture recognition AAC systems.)*
> > *Comment: **"to facilitate communication upon the following groups of
> > people"*: *"with"* or *"by"* rather than *"upon"*?
> > *
> > *The rating scale for the examples of *"people who experience . . .
> > "*(list of descriptions) is:
> > Highly Suitable Quite Suitable Neither Suitable or Useless Quite
> > Useless Highly
> > Useless
> > I would really like an option to say that I cannot comment on an example
> > description rather than have to choose "*Neither Suitable or Useless*". (I
> > assume this option should read *"Neither Suitable NOR Useless"*.)
> > Perhaps I am having difficulty with some aspects of the the survey because
> > my experience of using visual gesture with AAC users is rather limited and
> > primarily with people who acquired a severe speech problem in adulthood. I
> > used a "pick and mix" approach: first, identifying the minimum number of
> > gestures needed to help clarify ambiguous speech sounds; next, narrowing
> > down further based on preference and ease of production for the
> > individual. For example, in the few instances where I have used this
> > approach, I found that the sign for "m" in one-handed fingerspelling was
> > preferred over the equivalent signs in Cued Speech and Cued Articulation.
> > This was because: the handshape looks more like the letter "m" than the
> > other two and learning and reinforced using drawings to emphasise the
> > similarity between the handshape, the letter "m" and the shape of the upper
> > lip.
> > I hope these comments are helpful,
> > *
> > Liz Panton MRCSLT*
> > On 12 April 2012 23:27, Jenny Herd <je...@herdy.org.uk> wrote:
> > > I just don;t quite understand how a gesture recognition system would help
> > > an AAC user....are you looking at whether Jolly phonics can be used to
> > > basically "spell" out words? Maybe it is me but i just dont quite follow
> > > at all.
> > > Sorry
> > > On 12/04/2012 16:33, Christopher Amery wrote:
Many thanks for your explanations Chris - I also found your answers to questions posted by others very helpful too.
Amanda is right in saying that many AAC users would find it difficult to produce visual gestures. However, I have met some people with both severe speech problems and severe physical disabilities who already used visual gesture systems at a phoneme level (more or less) to augment their speech successfully. The examples that come to mind are:
- Percy-Hedley Foundation students who had been taught the Paget-Gorman signed speech system;
- a young woman and a young man who I met in Scotland who both signed using one-handed finger-spelling, if I remember rightly, with arm lowered and outstretched alongside the wheelchair (Sally Miller can correct me if my memory is wrong here);
- a young man from Iran with severe chorea-form movements who used ASL and finger-spelling with much better accuracy than might be expected in terms of fine motor dexterity, but whose signing was challenging to follow due to continual and unpredictable whole-body movements, so the placement of individual signs occurred randomly, wherever his arms and hands might be at the time.
The main drawback in all these cases was that the communication partner needed to be well-versed in the specific sign system used, including idiosyncrasies, or have a "translator" on hand. I struggle to imagine that a computerised system using visual recognition would be able to cope with some of these situations but can imagine that it should be feasible when visual gestures are always produced within a specific, limited space and are executed clearly enough to be differentiated by a "familiar communication partner".
Another other key issue is that the incorporation of a visual gesture system into a multi-modal communication system might have come about because of a preference NOT to use aided communication, at least in some circumstances. This might be because it is so much faster and direct, analogous to the situations where experienced and fluent "ETRAN-users" (ie. users on both sides of the frame) can dispense with the visual aid to eye-movement decoding.
Putting aside the practicalities, a computerised "visual gesture decoder" might be preferred by some people as an "interpreter", ie. reading and translating the communication with the partner, rather than having to use a device that requires you to interact in specific way with the "machine" rather than another person, eg. by physical manipulation of some sort, or gaze-selection of items displayed.
The project mentioned by Pat Sweet (GREAT: Gesture REcognition in Aphasia Therapy http://www.soi.city.ac.uk/great/?page_id=244 ) is very interesting and there has been a welcome resurgence of interest and research into aphasia and gesture - my weapon of choice being AmerInd, I am always on the look-out for developments in this field.
Given the related developments, I guess you are left with your original question, which was specific to visual gesture and phonology, so my apologies for straying a bit far off that track . . .
> This may be a bit of tangent for your research, but a team at City > University has been looking at how to use computerised training programmes > to teach gesture as a means of communication. They are working with adults > who have aphasia, usually as a result of stroke. The computer software they > have developed teaches specific gestures (eg 'umbrella' or 'glasses') and > also provides virtual environments where people can respond to what is > happening via gesture.
> Reg. charity Number: 1061636 > FAST is a limited company no. 3331184 (England and Wales)
> Registered Address: FAST, c/o King Loose & Co, St John's House, 5 South > Parade, Summertown, Oxford OX2 7JL
> To stay up to date with the latest news on assistive technology why not > subscribe to FAST's free monthly e-newsletters. Send an email to > i...@fastuk.org to subscribe to AT Policy News. To subscribe to AT > Research and Development News send an email to resea...@fastuk.org.
> fastsmall
> -----Original Message----- > From: cm-aac-forum@googlegroups.com [mailto:cm-aac-forum@googlegroups.com] > On Behalf Of Christopher Amery > Sent: 13 April 2012 12:39 > To: CM AAC Forum > Subject: [AAC Forum] Re: Student Needing Further Help
> Hey Liz, thanks for your comprehensive post. You've brought up a > couple of good points.
> In your response to question 1, that's exactly how I picture it. This > is a system that would 'decode' gestures that represent Jolly-Phonics > and allow an AAC user to blend and construct different phonemes > together in order to make a word. The system would produce this word > in the output form such as synthetic speech.
> In your response to question 2, there is potential for vision based > recognition systems to be 'diversified' for other uses. While my > primary aim is to construct a prototype system which recognises Jolly- > Phonic phonemes and blend these together to output words, the > technical and mechanical workings of the software for the ability to > recognise and associate gestures could be adapted to recognise other > gestures, such as Cued Speech or Cued Articulation gestures. This > question simply aims to evaluate whether these alternative systems > such as Cued Speech or Cued Articulation would benefit more from a > visual recognition system.
> In response to question 3, this is a limitation of Google Documents, > where only five choices can be added. I'll try and amend it > accordingly though.
> However, your suggestion to include a comment box for alternative > systems which i've not picked up on is something i'll include > momentarily.
> Thanks, Chris
> On Apr 13, 12:23 pm, "Liz Panton <l...@panton.me.uk>" > <l...@panton.me.uk> wrote: > > Hi Christopher,
> > I have had a go at completing the survey and had a look some of the > > previous messages from when you first posted here in February.
> > I have not "returned" the survey because you might still be revising it > and > > I thought it might help to post comments here rather than in the > "Comments" > > section of the survey itself. (I am assuming that once a survey form has > > been completed and returned then it would not be possible to complete it > a > > second time.)
> > *Q1. **How important do you value the following functionality or features > > for a technological AAC intervention involved in recognising visual > > gestures?*
> > *Comment:* My replies to this question would be based on imagining that > the > > device would recognise the visual gestures made by the AAC-user, in order > > to assist the communication partner to understand the message. This > would > > be similar in concept to an device that "decoded", for example, BSL or > > Makaton or Fingerspelling production for the benefit of a communication > > partner who did not understand these systems. Am I right in thinking that > > this is how you envisage the device being used? Or, if not, that this is > > still a way in which the device could be used?
> > *Q2. How much value do you place on the ability to utilise the following > > phonological systems for an AAC intervention specialising in visual > > recognition? (This question aims to evaluate the potential suitability > for > > alternative systems to be utilised to facilitate gestures.)*
> > *Comment:* The systems listed are Jolly-Phonics, Cued Articulation and > Cued > > Speech. I thought I understood this question correctly at first but > then I > > noticed the explanation, "*This question aims to evaluate the potential > > suitability for alternative systems to be utilised to facilitate > > gestures"*and now I am not so sure. What I thought you had in mind > > was a device that > > aimed to facilitate communication using gestures, by "decoding" the > > gestures, rather than a device that facilitated the gestures themselves. > > Did you mean to say that the device would aim to *"facilitate > communication > > using gestures"*?
> > The options for Q1 include personalisation of the device but for Q2 are > > restricted to the three named systems. The following comment is based on > > my expectation, perhaps wrong, that you are considering a device that > might > > *"facilitate communication using gestures":*
> > I appreciate that your starting point was Jolly-Phonics but, having > widened > > the questions to include Cued Speech and Cued Articulation, I wonder if > it > > might be helpful to leave space for respondents to add other systems that > > might be used in a similar way, ie. production of other hand shapes close > > to the face while speaking? I say this because I am aware that one-handed > > fingerspelling is sometimes used in this way, although it is not a > > *"phonological > > system"* of course.
Hi Liz - you explained that brilliantly and made the point about variability of gesture far more clearly than my attempt. Good luck with your research Chris and thanks for your explanations. Amanda
Sent from my iPhone
On 15 Apr 2012, at 18:09, "Liz Panton <l...@panton.me.uk>" <l...@panton.me.uk> wrote:
> Many thanks for your explanations Chris - I also found your answers to questions posted by others very helpful too.
> Amanda is right in saying that many AAC users would find it difficult to produce visual gestures. However, I have met some people with both severe speech problems and severe physical disabilities who already used visual gesture systems at a phoneme level (more or less) to augment their speech successfully. The examples that come to mind are: > Percy-Hedley Foundation students who had been taught the Paget-Gorman signed speech system;
> a young woman and a young man who I met in Scotland who both signed using one-handed finger-spelling, if I remember rightly, with arm lowered and outstretched alongside the wheelchair (Sally Miller can correct me if my memory is wrong here);
> a young man from Iran with severe chorea-form movements who used ASL and finger-spelling with much better accuracy than might be expected in terms of fine motor dexterity, but whose signing was challenging to follow due to continual and unpredictable whole-body movements, so the placement of individual signs occurred randomly, wherever his arms and hands might be at the time.
> The main drawback in all these cases was that the communication partner needed to be well-versed in the specific sign system used, including idiosyncrasies, or have a "translator" on hand. I struggle to imagine that a computerised system using visual recognition would be able to cope with some of these situations but can imagine that it should be feasible when visual gestures are always produced within a specific, limited space and are executed clearly enough to be differentiated by a "familiar communication partner".
> Another other key issue is that the incorporation of a visual gesture system into a multi-modal communication system might have come about because of a preference NOT to use aided communication, at least in some circumstances. This might be because it is so much faster and direct, analogous to the situations where experienced and fluent "ETRAN-users" (ie. users on both sides of the frame) can dispense with the visual aid to eye-movement decoding.
> Putting aside the practicalities, a computerised "visual gesture decoder" might be preferred by some people as an "interpreter", ie. reading and translating the communication with the partner, rather than having to use a device that requires you to interact in specific way with the "machine" rather than another person, eg. by physical manipulation of some sort, or gaze-selection of items displayed.
> The project mentioned by Pat Sweet (GREAT: Gesture REcognition in Aphasia Therapy http://www.soi.city.ac.uk/great/?page_id=244 ) is very interesting and there has been a welcome resurgence of interest and research into aphasia and gesture - my weapon of choice being AmerInd, I am always on the look-out for developments in this field.
> Given the related developments, I guess you are left with your original question, which was specific to visual gesture and phonology, so my apologies for straying a bit far off that track . . .
> On 13 April 2012 13:56, Pat Sweet <P...@fastuk.org> wrote: > Hi Chris,
> This may be a bit of tangent for your research, but a team at City University has been looking at how to use computerised training programmes to teach gesture as a means of communication. They are working with adults who have aphasia, usually as a result of stroke. The computer software they have developed teaches specific gestures (eg 'umbrella' or 'glasses') and also provides virtual environments where people can respond to what is happening via gesture.
> Reg. charity Number: 1061636 > FAST is a limited company no. 3331184 (England and Wales)
> Registered Address: FAST, c/o King Loose & Co, St John's House, 5 South Parade, Summertown, Oxford OX2 7JL
> To stay up to date with the latest news on assistive technology why not subscribe to FAST's free monthly e-newsletters. Send an email to i...@fastuk.org to subscribe to AT Policy News. To subscribe to AT Research and Development News send an email to resea...@fastuk.org.
> fastsmall
> -----Original Message----- > From: cm-aac-forum@googlegroups.com [mailto:cm-aac-forum@googlegroups.com] On Behalf Of Christopher Amery > Sent: 13 April 2012 12:39 > To: CM AAC Forum > Subject: [AAC Forum] Re: Student Needing Further Help
> Hey Liz, thanks for your comprehensive post. You've brought up a > couple of good points.
> In your response to question 1, that's exactly how I picture it. This > is a system that would 'decode' gestures that represent Jolly-Phonics > and allow an AAC user to blend and construct different phonemes > together in order to make a word. The system would produce this word > in the output form such as synthetic speech.
> In your response to question 2, there is potential for vision based > recognition systems to be 'diversified' for other uses. While my > primary aim is to construct a prototype system which recognises Jolly- > Phonic phonemes and blend these together to output words, the > technical and mechanical workings of the software for the ability to > recognise and associate gestures could be adapted to recognise other > gestures, such as Cued Speech or Cued Articulation gestures. This > question simply aims to evaluate whether these alternative systems > such as Cued Speech or Cued Articulation would benefit more from a > visual recognition system.
> In response to question 3, this is a limitation of Google Documents, > where only five choices can be added. I'll try and amend it > accordingly though.
> However, your suggestion to include a comment box for alternative > systems which i've not picked up on is something i'll include > momentarily.
> Thanks, Chris
> On Apr 13, 12:23 pm, "Liz Panton <l...@panton.me.uk>" > <l...@panton.me.uk> wrote: > > Hi Christopher,
> > I have had a go at completing the survey and had a look some of the > > previous messages from when you first posted here in February.
> > I have not "returned" the survey because you might still be revising it and > > I thought it might help to post comments here rather than in the "Comments" > > section of the survey itself. (I am assuming that once a survey form has > > been completed and returned then it would not be possible to complete it a > > second time.)
> > *Q1. **How important do you value the following functionality or features > > for a technological AAC intervention involved in recognising visual > > gestures?*
> > *Comment:* My replies to this question would be based on imagining that the > > device would recognise the visual gestures made by the AAC-user, in order > > to assist the communication partner to understand the message. This would > > be similar in concept to an device that "decoded", for example, BSL or > > Makaton or Fingerspelling production for the benefit of a communication > > partner who did not understand these systems. Am I right in thinking that > > this is how you envisage the device being used? Or, if not, that this is > > still a way in which the device could be used?
> > *Q2. How much value do you place on the ability to utilise the following > > phonological systems for an AAC intervention specialising in visual > > recognition? (This question aims to evaluate the potential suitability for > > alternative systems to be utilised to facilitate gestures.)*
> > *Comment:* The systems listed are Jolly-Phonics, Cued Articulation and Cued > > Speech. I thought I understood this question correctly at first but then I > > noticed the explanation, "*This question aims to evaluate the potential > > suitability for alternative systems to be utilised to facilitate > > gestures"*and now I am not so sure. What I thought you had in mind > > was a device that > > aimed to facilitate communication using gestures, by "decoding" the > > gestures, rather than a device that facilitated the gestures themselves. > > Did you mean to say that the device would aim to *"facilitate communication > > using gestures"*?
> > The options for Q1 include personalisation of the device but for Q2 are > > restricted to the three named systems. The following comment is based on > > my expectation, perhaps wrong, that you are considering a device that might > > *"facilitate communication using gestures":*
> > I appreciate that your starting point was Jolly-Phonics but, having widened > > the questions to include Cued Speech and Cued Articulation, I wonder if it > > might be helpful to leave space for respondents to add other systems that > > might be used in a
