[Standards] Advice on XEP-0301 Mass-Market Use Cases (if XEP-0301 added to Adium/Miranda within 12 months)

[Mark Rejhon]

To the Stadards group, 

I bring up a question separate from the Session Handling.

(many questions have already been answered -- thank you very much)

Scenario: XEP-0301 real-time text gets added to a mainstream client such as Pidgin or Miranda within 12 months.

Question: What is the simplest possible and easiest method of safely introduce real-time text politely to a mainstream client, in a non-intrusive manner? (i.e. privacy) 

Especially if one end has RTT enabled by default (i.e. intentionally enabled by user), and the other end has RTT disabled by default (i.e. default installation)

Details of Scenarios:

- People don't normally expect their typing to be transmitted in real time.  Therefore, feature shold be off by default in such mainstream clients.

- However, it should be easy as possible to enable.  Therefore, such mainstream clients will always advertise XEP-0301 compatibility via disco (section 5) but not necessarly enable the transmission of outgoing XEP-0301, including for privacy reasons.

- Some people, such as deaf users, will want to change the software preference to permanently allow XEP-0301.  

(i.e. in software preferences)   

- However, even when disabled by default, we want users to make it easy for XEP-0301-aware recipients to enable the feature for responding to other XEP-0301 clients.

(i.e. per-chat-session activation feature, such as a button)

Potential popularity in 10 years

Just like all television sets have a closed caption decoder, and is usually turned off by default, but easily turned on -- eventually, we hope XEP-0301 becomes mainstream within 10 years, as a replacement for obsolete deaf TDD/TTY teletypewriters.   In even my limited testing (non-deaf family, friends, coworkers, etc), I found that once they understood what RTT was, it was a more popular feature than audio and video.   So RTT has a lot of potential to improve the IM experience, even if it's an option feature enabled only a few percent of the time.   Although more scientific surveys need to be done once it's already a part of Adium/Miranda/etc, it shows that sometimes people like to switch to the conversational mode via RTT rather than via audio, since they can talk very conversationally without making noise.   Other times, people are in an asynchronous mood, like text messaging, regular instant messages, or email.   But it apparently shows that there's more frequently a mood to be conversational via RTT than via audio or video, from this limited testing.   Thus, we need to be prepared for the potential possibility that it may show up in mainstream clients.

NOTE: We've also got a new logo for real-time text at www.fasttext.org which we'll be starting to use in the next 12 months for future programming.

Advertising XEP-0301 capability is done by a caps detection.  Presumably, this will always be the case in all clients that supports XEP-0301.

We see three situations where chat is activated between two XEP-0301 aware clients:

1. Both clients advertise XEP-0301 capability, but does not send XEP-0301 by default.

2. Both clients advertise XEP-0301 capability, but only recipient sends XEP-0301 by default.

3. Both clients advertise XEP-0301 capability, but only sender sends XEP-0301 by default.

We want to make it easy for both ends to start sending RTT in scenario 2 and 3, even if disabled by default.

Example "Accessible" future use case (which will happen!):

-- Alice is a deaf user, using a year 2014 build of Adium.  She has XEP-0301 enabled.

-- Bob is a hearing user, not aware of real-time text, but is using a year 2014 build of Pidgin that has XEP-0301 support but does not transmit RTT by default.

-- Alice sends a message to Bob.  She immediately sends RTT.  Her typing is transmitted in real time.

-- Bob's client software detects this but does not automatically send Bob's typing in real time.

The big question: What should happen afterwards?

Possible Ideas/Methods of negotiating RTT activation

- XEP-0020?  Bob instant messaging client, should be able to activate a session negotiation feature.  As you remember, we added an XEP-0020 session negotiation feature in Version 0.0.1 of XMPP-RTT, but it got removed from Version 0.0.2 of XMPP-RTT.   It is overkill/complesx.

- Jingle?  I think it's still overkill, since we want to operate 100% in-band

- XEP-0030?  We should always advertise XEP-0301 even if the client doesn't send outgoing RTT by default.  This allows clients that DO WANT to send RTT, to go ahead and send RTT anyway

- XEP-0085?  Nope, it is not suitable.

- The existing XEP-0301 featuer of event='start' and event='cancel'?  Yes, this is a possibility, but if I can simplify things by removing these... (as long as I have an alternative to cover the above use case.)

The simplest method I have though of, to cover such use case:

1. Alice's software verifies Bob's softtware does support RTT via caps detection. (Section 5 of XEP-0301)

2. Alcie Send RTT. 

3. Bob's client, upon detecting incoming <RTT> automatically displays a message

*** Alice is sending real-time text.  Click [Activate] now to also send your typing live in real-time, too!

4. Bob clicks "Activate", realizing that he's watching Alice's typing live.  

(Note: Bob can continue to send line-by-line instant messages asking what real-time text is, etc, before clicking the Activate button, etc.)

5. Bob is now sending real-time text even though his RTT was disabled by default in his default install of his mass-market instant messaging software.

We need to emphasive RTT is an accessibility issue too -- that it may be available but disabled in mass-market software clients.

So I want to hear from the Standards group, about the ideal usage cases for XEP-0301 in a mass-market client.   It's an interoperability issue, because if we have multiple different standards of negotiating activation of RTT, then it does not work very well.   

At the same time, we need:

-- Keep it simple as possible, while staying accessible

-- Minimum amount of protocol overhead, while staying accessible

-- Keep it in-band, while staying accessible

Has anybody else found an even simpler method, or a reason to use a more complex activation method for RTT?

Thanks,

Mark Rejhon

[Bernard Aboba]

With respect to the Alice/Bob exchange, is there an expectation that the RTT preference would persist?  Or is it just a choice for that particular conversation?

There is also the MUC scenario.  How does discovery fit into this case?  It also seems that there will be some rooms where RTT would want to be on by default (e.g. emergency use case). 

[Mark Rejhon]

Hello Bernard, 

Yes -- 

There would likely ideally be both a per-session activation mechanism, 

as well as a global auto-accept setting in the software's own preferences.

-- Mainstream clients (i.e. Adium, Pidgin) would presumably have auto-accept disabled (by default) in a default software installation, and likely provide an activation feature (a button, and/or a popup notification similar to "Real-text is available. Click enable to activate", etc.) when the software determines the other end is detected to support real-time text.   Users who always wants RTT enabled, can then go into preferences and turn on auto-accept.

-- Assistive clients targeted to the deaf market would presumably have auto-accept enabled in a default software installation, and not need to click a button to activate RTT.  Mainstream clients (autoaccept turned off) suddenly receiving RTT, would prompt the user if they want to enable sending back of RTT.

MUC easily accepts RTT, but the dynamics of activating RTT for a MUC can become significantly complicated.  For the interim (at the moment) we assume it's safe to just broadcast RTT into a MUC we know allows RTT (i.e. deaf chat rooms, next generation 9-1-1 experimental system, etc) because these are usually niche situations, with specialized clients pre-configured to specialized chat rooms.

If RTT becomes popular this topic will need to be revisted.   A one-on-one chat that already has RTT, that gets converted to MUC, would be assumed to continue RTT.   RTT works in a mixed-mode fashion (RTT clients and non-RTT clients) with group chat rooms, documented in:

http://www.marky.com/realjabber/XMPP-RTT-Supplement_2011-06-17.pdf

(supplemental document that covers discussion about RTT in MUC)

Right now, the email I wrote is chiefly targeted at non-MUC situations, on the best-practices.  I think I found a situation that will be 100% compatible with MUC.

There has been further thought and internal discussion on the matter -- 

Thanks,

Mark Rejhon

[Kevin Smith]

On Wed, Feb 29, 2012 at 3:06 AM, Mark Rejhon <mark...@gmail.com> wrote:
> Question: What is the simplest possible and easiest method of safely
> introduce real-time text politely to a mainstream client, in a non-intrusive
> manner? (i.e. privacy)

I think the simplest way that satisfies most requirements is a pair of
switches in a client:
1) Request RTT sessions (default false)
2) Provide RTT session if requested (maybe default true)

I'm not yet sure if this is entirely sufficient, but it's pretty simple.

/K

[Mark Rejhon]

On Fri, Mar 2, 2012 at 8:45 AM, Kevin Smith <ke...@kismith.co.uk> wrote:

I think the simplest way that satisfies most requirements is a pair of

switches in a client:
1) Request RTT sessions (default false)
2) Provide RTT session if requested (maybe default true)

I'm not yet sure if this is entirely sufficient, but it's pretty simple.

This may be sufficient for mention in the spec (I do want to add at least a few sentences to Implementation Notes), but there are also the other pressing criteria:

* A technique that keeps it simple for the end-user

* Yet keep it easily accessible. 

* Not everyone will want to view RTT

* Not everyone will want to send back RTT.

Some of these are UI considerations (how easy or difficult it is to enable), and some of them require some basic standards (in order to maintain interop)

I think it is essentially your recommendation, with a variant:

1) If RTT is supported, it is strongly RECOMMENDED to advertise RTT capability via disco#info (section 5 of XEP-0301)

2) Request RTT sessions 

Configurable Preference: per-session | always  

Default: per-session (in mass-market clients)

3) Accept RTT session 

Configurable Preference: auto-accept | prompt-user-confirm-first | reject

Default: prompt-user-confirm-first (in mass-market clients)

This essentially means, the following:

-- Senders: Requesting outgoing RTT session: 

Just simply transmit outgoing <rtt> (if recipient capability is available)

-- Recipients: Detecting incoming RTT session request: 

Recipient simply detects incoming <rtt>.  Recipient client SHOULD NOT display RTT until recipient also enables/confirms RTT.

-- In addition, when detecting remote RTT capability via disco#info, a small notification/indicator should happen.  An example is automatically enabling a button that needs to be activated per-session, and/or displaying a status message inside the Message History "*** Real-time text is available for this session.  Click to enable."

-- If  "prompt-user-confirm-first" is enabled, a confirmation message should show (preferably non-intrusive, such as one that pops up into the message history, "*** Sender is using real-time text. Click [Activate] to transmit your text live while you type.") ... Only one confirmation should show per chat session (i.e. until the a new session starts, or if a new window is opened).    Incoming <rtt> transmissions are ignored until real-time text is accepted/enabled.

This keeps RTT non-intrusive but very accessible and easy to enable.

This mechanism is also MUC compatible.

[Mark Rejhon]

On Fri, Mar 2, 2012 at 10:27 AM, Mark Rejhon <mark...@gmail.com> wrote:

On Fri, Mar 2, 2012 at 8:45 AM, Kevin Smith <ke...@kismith.co.uk> wrote:

I think the simplest way that satisfies most requirements is a pair of

switches in a client:
1) Request RTT sessions (default false)
2) Provide RTT session if requested (maybe default true)
I'm not yet sure if this is entirely sufficient, but it's pretty simple.

[snip]

I think it is essentially your recommendation, with a variant:

1) If RTT is supported, it is strongly RECOMMENDED to advertise RTT capability via disco#info (section 5 of XEP-0301)

2) Request RTT sessions 

Configurable Preference: per-session | always  

Default: per-session (in mass-market clients)

3) Accept RTT session 

Configurable Preference: auto-accept | prompt-user-confirm-first | reject

Default: prompt-user-confirm-first (in mass-market clients)

[snip]

On a related subject, I have been thinking of the important section 5 of XEP-0301:

http://xmpp.org/extensions/xep-0301.html#determining_support 

It would certainly surely simplify a lot of my programming if I could make it 100% self-contained into the <rtt/> element, because programming a real-time library (for things like strophe, libpurple, jabber-net, etc) is most easily done without worrying about using <iq> stanzas.

I am seriously considering making disco#info section 5 optional, and making REQUIRED an alternate backup capabilities-detection mechanism:

1. Clients that support XEP-0301, shall send one, single empty <rtt xmlns='urn:xmpp:rtt:0'/> element at the beginning of a chat session.  This does not indicate an invitation to establish an RTT session, but merely to advertise a capability.  No further <rtt> elements shall be transmitted unless the other client advertises RTT support too (either via disco#info or via having ever received <rtt>)

This simplifies the programming of real-time libraries for Adium, Pidgin, Miranda, and any other instant messaging clients -- because 100% of the XEP-0301 protocol is solely confined to <rtt/> elements.

Sincerely,
Mark Rejhon

[Kevin Smith]

On Fri, Mar 2, 2012 at 3:57 PM, Mark Rejhon <mark...@gmail.com> wrote:
> I am seriously considering making disco#info section 5 optional, and making
> REQUIRED an alternate backup capabilities-detection mechanism:

That's very not-XMPPish, though - you're not likely to need to do any
work with xep30 or 115, the library/client will already be
implementing that, will have a easy way to add a feature and to check
that a remote contact supports a feature.

/K

[Mark Rejhon]

On Fri, Mar 2, 2012 at 10:57 AM, Mark Rejhon <mark...@gmail.com> wrote:

I am seriously considering making disco#info section 5 optional, and making REQUIRED an alternate backup capabilities-detection mechanism:

1. Clients that support XEP-0301, shall send one, single empty <rtt xmlns='urn:xmpp:rtt:0'/> element at the beginning of a chat session.  This does not indicate an invitation to establish an RTT session, but merely to advertise a capability.  No further <rtt> elements shall be transmitted unless the other client advertises RTT support too (either via disco#info or via having ever received <rtt>)

This simplifies the programming of real-time libraries for Adium, Pidgin, Miranda, and any other instant messaging clients -- because 100% of the XEP-0301 protocol is solely confined to <rtt/> elements.

Correction:  "...because 100% of the /REQUIRED parts/ of the XEP-0301 protocol is solely confined to <rtt/> elements."

[Peter Saint-Andre]

+1

[Mark Rejhon]

Point taken -- I am leaving section 5 unmodified. 

I agree it is not very XMPP-ish, but there were a couple other rationales to the idea"

-- I remain concerned that not all platforms that I will be working on, provides an easy/trivial way to query for this feature.  I also fear that I may run into XMPP implementations that isn't transparent to <iq> but lets through unfamiliar stanzas (such as <message> <rtt> </message> etc.) ... Although that hasn't happened yet, I've seen some XMPP implementations filter unfamiliar stanzas (i.e. Facebook XMPP server).   

-- As long as section 5 is required, it means the server must have fully working <iq> transprency _AND_ transparency on <rtt/> elements.  Two potential weak links in a 'half-hearted' XMPP implementation.

However, this can be revisited for subsequent revisions, if it becomes a pressing need.  For now, I will attempt to comply with section 5 in all implementations, and see what the real-world experience is, first.

Mark Rejhon

[Kevin Smith]

On Fri, Mar 2, 2012 at 6:02 PM, Mark Rejhon <mark...@gmail.com> wrote:
> -- As long as section 5 is required, it means the server must have fully
> working <iq> transprency _AND_ transparency on <rtt/> elements.  Two
> potential weak links in a 'half-hearted' XMPP implementation.

A server that doesn't support <iq/> isn't coming anywhere close to
being a compliant XMPP implementation, though, and is likely to have
all sorts of other issues (like only passing the <body/> part of a
<message/> or similar japery).

/K

[Bernard Aboba]

Mark said:

-- I remain concerned that not all platforms that I will be working on, provides an easy/trivial way to query for this feature.  I also fear that I may run into XMPP implementations that isn't transparent to <iq> but lets through unfamiliar stanzas (such as <message> <rtt> </message> etc.) ... Although that hasn't happened yet, I've seen some XMPP implementations filter unfamiliar stanzas (i.e. Facebook XMPP server).   

-- As long as section 5 is required, it means the server must have fully working <iq> transparency _AND_ transparency on <rtt/> elements.  Two potential weak links in a 'half-hearted' XMPP implementation.

However, this can be revisited for subsequent revisions, if it becomes a pressing need.  For now, I will attempt to comply with section 5 in all implementations, and see what the real-world experience is, first.

[BA] I share your concern about potential odd behavior, but at the same time, it's probably not prudent to deviate from best practices without real-world experience.  When necessary, the transition from blissful naivete to cynical disappointment can be accomplished quite quickly :)