Re: spellchecking API

[JOSE MANUEL CANTERA FONSECA]

How the current language would be determined?

El 28/11/11 00:07, "Andreas Gal" <g...@mozilla.com> escribió:

>
>I started a thread on dev-platform w.r.t. to a spellchecking API. Lets
>discuss the precise API here. Bikeshed away.
>
>navigator.spelling.check("mispelled", callback)
>
>function callback(input, boolean)
>
>navigator.spelling.suggest("almosttright", callback)
>
>function callback(input, array-of-suggestioned words/sentence)
>
>In case of both APIs, entire sentence parts or sentences can be submitted.
>
>The implementation will optimize for repeated calls to the API with
>similar input (e.g. as text is added to an input field).
>
>The callback is always called, even if no dictionary could be found. In
>that case check returns "true" and suggest returns an empty set of
>suggestions. The callback can be async.
>
>Only webapps with the appropriate permission set can access the UA-wide
>personal dictionary. Sites can only use their own site-specific personal
>dictionary.
>
>Words can be added to/removed from the personal dictionary:
>
>navigator.spelling.recognize("word")
>navigator.spelling.forget("word")
>
>The personal dictionary can be queried with:
>
>navigator.spelling.enumerate(callback)
>
>function callback(array-of-words-or-sentence-parts)
>
>Again, sites can only query their own personal dictionary.
>
>Andreas
>_______________________________________________
>dev-webapi mailing list
>dev-w...@lists.mozilla.org
>https://lists.mozilla.org/listinfo/dev-webapi


Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo.
This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at.
http://www.tid.es/ES/PAGINAS/disclaimer.aspx

[Andreas Gal]

navigator.language is the default, but we should probably add an optional language parameter so its possible to spellcheck for a different language than the default

Andreas

[Ehsan Akhgari]

Hey Andreas,

Have you seen <
http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/0516.html>?
Its checkWord and getSuggestionForWord methods seem to be very similar to
your proposal (although it doesn't address the issue of personal
dictionaries...)

I've talked to WebKit folks about this before, and they seem to be
interested in implementing this API. So I think we should perhaps try to
see if we can agree on an API which both engines would be willing to
implement. I'm CCing hbono who proposed that API, and also Ryosuke who
hacks on WebKit editing component.

Cheers,
--
Ehsan
<http://ehsanakhgari.org/>

[Andreas Gal]

Great. Maybe we should try to harmonize the proposal and propose it jointly.

Morrita, can you check whether the API below makes sense for groups of words or entire phrases. This was pointed out as important for many languages for high quality spell checking.

As for add/removeMarker, I think thats a mistake to add in this form. Squiggly underlines aren't in CSS, we should fix that instead, and then a marker API isn't needed (in the meantime its possible to use a background image with a repeat pattern).

The other two methods seem almost identical indeed.

hbono, what do you think?

As described below I think this is a pretty simple implementation and I am happy to commit someone to implementing it, in particular if Google would do the same in parallel.

Andreas

On Nov 28, 2011, at 8:31 PM, Ryosuke Niwa wrote:

> Hi Ehsan & Andreas,
>
> It's great to see some traction in this area! I'm more than happy to have a discussion on public-webapps or have an in-person meeting about this.
>
> +morrita who's a WebKit/Chromium's spellcheck guru.
>
> - Ryosuke

[Robert O'Callahan]

On Tue, Nov 29, 2011 at 10:04 AM, Andreas Gal <g...@mozilla.com> wrote:

> As for add/removeMarker, I think thats a mistake to add in this form.
> Squiggly underlines aren't in CSS, we should fix that instead, and then a
> marker API isn't needed (in the meantime its possible to use a background
> image with a repeat pattern).
>

It's actually pretty difficult to use CSS text-decorations in the general
case where a word can span DOM boundaries in annoying ways, e.g.
<span>Hello Ki</span>tty.

Something more like ::selection, that allows the Web app to define
arbitrary named selections and style them, would be a lot better here.

Rob
--
"If we claim to be without sin, we deceive ourselves and the truth is not
in us. If we confess our sins, he is faithful and just and will forgive us
our sins and purify us from all unrighteousness. If we claim we have not
sinned, we make him out to be a liar and his word is not in us." [1 John
1:8-10]

[Andreas Gal]

Supporting grammar and spelling errors is definitely a good idea. I like the new API. We can fine-tune it as we go along. We should file bugs for webkit and Gecko to get an implementation going. I will find someone to own this on our end.

Adding dbaron and bz to discuss the marking/highlighting API. A separate more generic API sounds like the way to go. david, bz, what do you think?

Andreas

> So, considering Robert's point, something like this may be a possible
> starting point for this discussion:
>
>
> [Supplemental=Document]
> interface MarkableDocument {
> // "className" specifies the name of the pseudo class.
> // Spelling markers should have some pre-defined names.
> DocumentMarker addMarker(Range range, DOMString className);
> void removeMarker(DocumentMaerker marker);
> void removeMarkers(Range range);
> sequence<DocumentMarker> getMarkersByRange(Range range);
> };
>
> interface DocumentMarker {
> // this range should be a snapshot. The modification won't reflect
> to the marker.
> readonly attribute Range range;
> readonly attribute DOMString className;
> };
>
>
> In addition to this, we also need to extend some on CSS side.
>
> For <input>, Bono-san is considering "virtal node" concept for his IME
> related work,
> which exposes the shadow dom via Node-ish (or temporal node) object.
> Spellcheck can borrow similar idea:
>
>
> [Supplemental=HTMLInputElement, HTMLTextareaElement]
> MarkableElement {
> readonly attribute Text markableText;
> };
>
>
> For marking the shadow DOM, we can create ranges from this
> "markableText" object,
> and pass them to the MarkableDocument API.
>
> Although actual shadow for <input> may be built from DOM subtree instead
> of single Text object, abstracting it as a Text would be reasonable
> abstractioin for our usecase. Bono-san will have some opinion here though.
> What do you think?
>
> Anyway, thanks for looping me in.
> Regards,
>
> --
> morrita

[Ryosuke Niwa]

[Hajime Morrita]

Hi Andreas,
Thanks you for your interest of this topic! Bono-san (hbono@) and I
are working on spellchecking stuff andWe're really happy to discuss
this and make a harmonized proposal.

As you mentioned, there are two types of API around spellchecking.Let
me call them "Spelling" API and "Marker" API here to avoid consfusion.
For Spelling API, I like your async style.
Note that currently WebKit uses blocking API, which is almost
directmappings of NSSpellChecker (on Mac) or Hunspell (on Chromium).
But weare working for making them async to improve UI responsiveness.
On API granuality, I agree that check() and suggest() is
sufficiententry point for our purpose. But I'm not sure if the boolean
returnvalue is always enough.
 * This implies that the page author need to tokenize text into words
 and call the API for each word. Such tokenization is not trivial in
CJK languages.
 * Sometimes we need to check large chunk of text. (We need it when
text is pasted for example.)  In that case, batch-style API,   similar
to NSSpellChecker#requestCheckingOfString(), would be more
efficient, compared to multiple API calls. This is especially true
for Chromium and WebKit2 which need an IPC for each API call.
 * WebKit (on Mac) supports grammar checking in addition to word based
 spell checking. So it would be great if the Web facing API also
supports that.
Based on points above, one alternative I can imagine is something like this:

interface SpellCheckerResult {  const DOMString MISSPELLING = ...;
const DOMString GRAMMAR = ...;  readonly attribute long offset;
readonly attribute long length;  readonly attribute DOMString type;};
[Callback]interface SpellCheckerCheckHandler {  void handle(DOMString
text, sequence<SpellCheckerResult>? results);};
interface SpellChecker {  void check(DOMString text,
SpellCheckerCheckHandler callback);  void suggest(DOMString
misspelledText, ...); // This would be same as your proposal.};
[Supplemental=Navigator]interface SpellCheckerHodler {  SpellChecker spelling;};

I think this is basically what you are proposing, except this returnsa
sequence of the result objects instead of single boolean.

On Marker API, I think we need something like this because...
 * As Robert pointed, marked ranges don't have associated DOM entity.
 It can spans multiple nodes. Thus specifying it via CSS wouldn't be
trivial.
 * Another tricky part is <input> and <textarea>. The text inside
these textfields don't have visible DOM nodes. (In WebKit, it's in
the shadow tree. I guess Gecko uses similar technique.)   Because of
this, specifying substring in that text looks difficult using CSS.
On the other hand, I totally agree that generalizing the markerconcept
beyond spell checking would be better.

So, considering Robert's point, something like this may be a

possiblestarting point for this discussion:

[Supplemental=Document]interface MarkableDocument {  // "className"
specifies the name of the pseudo class.   // Spelling markers should
have some pre-defined names.  DocumentMarker addMarker(Range range,
DOMString className);   void removeMarker(DocumentMaerker marker);
void removeMarkers(Range range);  sequence<DocumentMarker>
getMarkersByRange(Range range);};
interface DocumentMarker {  // this range should be a snapshot. The
modification won't reflect to the marker.  readonly attribute Range
range;   readonly attribute DOMString className;};

In addition to this, we also need to extend some on CSS side.
For <input>, Bono-san is considering "virtal node" concept for his IME

related work,which exposes the shadow dom via Node-ish (or temporal
node) object.Spellcheck can borrow similar idea:

[Supplemental=HTMLInputElement, HTMLTextareaElement]MarkableElement {
readonly attribute Text markableText;};

For marking the shadow DOM, we can create ranges from this

"markableText" object,and pass them to the MarkableDocument API.

Although actual shadow for <input> may be built from DOM subtree

insteadof single Text object, abstracting it as a Text would be
reasonableabstraction for our use-case. What do you think? (And
Bono-san will also have some opinion here.)
Thanks for looping me in anyway.

--morrita

--
morrita

[Hajime Morrita]

Resending because GMail ate my carriage returns....

----

Hi Andreas,

Thanks you for your interest of this topic!
Bono-san (hbono@) and I are working on spellchecking stuff and
We're really happy to discuss this and make a harmonized proposal.


As you mentioned, there are two types of API around spellchecking.

Let me call them "Spelling" API and "Marker" API here to avoid confusion.

[Hironori Bono (坊野 博典)]

Greetings Morrita-san,

Thank you for your detailed explanation.
Even though my WebKit change <http://webkit.org/b/59693> used some new
types (such as SpellcheckResult) for implementing my Spellcheck API, I
think it may be better to use a text node instead of DOMString so we
can use Range to specify the misspelled text as listed in the
following example:

var node = document.createTextNode('This is a test.');
SpellCheckerHolder.spelling.check(node, function(node, results) {
for (var i = 0; i < results.length; ++i) {
var misspelledText = results[0].range.toString();
SpellCheckerHolder.spelling.suggest(misspelledText, ...);
}
}

I have updated your IDLs to change some parameters.

interface SpellCheckerResult {
const DOMString MISSPELLING = ...;
const DOMString GRAMMAR = ...;

readonly attribute Range range;

readonly attribute DOMString type;
};

[Callback]
interface SpellCheckerCheckHandler {

void handle(Node node, sequence<SpellCheckerResult>? results);
};

interface SpellChecker {
void check(Node node, SpellCheckerCheckHandler callback);

void suggest(DOMString misspelledText, ...); // This would be same
as your proposal.
};

Regards,

Hironori Bono
E-mail: hb...@google.com

[Ehsan Akhgari]

Hi all,

I think SpellChecker is a great interface which is capable of handling both
spelling and grammar checking, and the fact that it can be implemented
asynchronously makes it a really good choice in face of different types of
spelling back-ends.

I'm not quite comfortable with the DocumentMarker interface, though. I
think that it's a bad idea to specify a className for a range, since
classNames apply to DOM elements currently, and making them "apply" to
ranges in this way would be confusing. For regular content, I think we
want a pseudo-element similar to ::selection <
http://www.w3.org/TR/2005/WD-css3-selectors-20051215/> (note that this has
been removed from the CSS3 Selectors TR, but I'm not aware of the details
on why that is) which matches multiple marked ranges of a document. We can
restrict the set of CSS properties honored by this pseudo-element to color,
background, text-decoration, and outline (plus perhaps other useful
properties).

The MarkableDocument interface will then look like below:

[Supplemental=Document]
interface MarkableDocument {
// this range should be a snapshot. The modification won't update the
marker.
void addMarker(Range range);
void removeMarker(Range marker);
sequence<Range> getMarkers();
};

And the CSS ::marked pseudo-element would match the markers associated with
a document.

(Note that this proposal does not allow support for different marker types,
so we might want to make this a bit less generic.)

For input and textarea, we need an entirely different solution, as their
contents are not accessible using regular DOM tree operations. The
proposed MarkableElement lacks support for multiple misspelling ranges, so
we would need something better. Here is my proposal:

interface MarkedSection {
unsigned long start;
unsigned long end;
};

[Supplemental=HTMLInputElement, HTMLTextareaElement]
interface MarkableElement {
void addMarker(MarkedSection section);
void removeMarker(MarkedSection section);
sequence<MarkedSection> getMarkers();
};

The MarkedSection interface is semantically compatible with the existing
selectionStart/selectionEnd properties of these elements. In order to
style the marked sections, we would need a pseudo-class (named :marked or
something similar) which would only apply to input and textarea elements,
and which would also honor the same properties as the ::marked
pseudo-element.

I'd be interested to know what you all think about this proposal for the
marking part.

Cheers,
--
Ehsan
<http://ehsanakhgari.org/>

On Tue, Nov 29, 2011 at 2:49 AM, Andreas Gal <g...@mozilla.com> wrote:

>
> Supporting grammar and spelling errors is definitely a good idea. I like
> the new API. We can fine-tune it as we go along. We should file bugs for
> webkit and Gecko to get an implementation going. I will find someone to own
> this on our end.
>
> Adding dbaron and bz to discuss the marking/highlighting API. A separate
> more generic API sounds like the way to go. david, bz, what do you think?
>
> Andreas
>

[L. David Baron]

On Tuesday 2011-11-29 08:49 +0100, Andreas Gal wrote:
>
> Supporting grammar and spelling errors is definitely a good idea. I like the new API. We can fine-tune it as we go along. We should file bugs for webkit and Gecko to get an implementation going. I will find someone to own this on our end.
>
> Adding dbaron and bz to discuss the marking/highlighting API. A separate more generic API sounds like the way to go. david, bz, what do you think?

The plan I've had for a long time for a marking API is described in
https://bugzilla.mozilla.org/show_bug.cgi?id=256773 . I can't load
the bug right now since bugzilla's down, but I don't think it's
described in a huge amount of detail there.

-David

--
𝄞 L. David Baron http://dbaron.org/ 𝄂
𝄢 Mozilla http://www.mozilla.org/ 𝄂

[Ehsan Akhgari]

On Tue, Nov 29, 2011 at 12:37 PM, L. David Baron <dba...@mozilla.com> wrote:

> On Tuesday 2011-11-29 08:49 +0100, Andreas Gal wrote:
> >
> > Supporting grammar and spelling errors is definitely a good idea. I like
> the new API. We can fine-tune it as we go along. We should file bugs for
> webkit and Gecko to get an implementation going. I will find someone to own
> this on our end.
> >
> > Adding dbaron and bz to discuss the marking/highlighting API. A separate
> more generic API sounds like the way to go. david, bz, what do you think?
>
> The plan I've had for a long time for a marking API is described in
> https://bugzilla.mozilla.org/show_bug.cgi?id=256773 . I can't load
> the bug right now since bugzilla's down, but I don't think it's
> described in a huge amount of detail there.
>

That's mostly about different selection types, right? I'm not sure if
that's a path we want to follow, because of the issue of multi-range
selections. Currently only Gecko support multi-range selections (sort of),
and they've already been removed from the Selection spec, and other engines
are unwilling to implement them. In fact, I seriously think that Gecko
should drop support for multi-range selection as well in the future (I have
a half-finished proposal for that around somewhere). Given all this, I
don't think that would be a good solution for the problem at hand.

--
Ehsan
<http://ehsanakhgari.org/>

[L. David Baron]

I don't see how it's connected to multi-range selections. The idea
is to provide a DOM API to associate a range with a named category
of selections, and then have CSS syntax for styling those named
categories that works just like ::selection / ::-moz-selection.

-David

[L. David Baron]

On Tuesday 2011-11-29 11:58 -0500, Ehsan Akhgari wrote:
> I'm not quite comfortable with the DocumentMarker interface, though. I
> think that it's a bad idea to specify a className for a range, since
> classNames apply to DOM elements currently, and making them "apply" to
> ranges in this way would be confusing.

I think we do need multiple names for ranges here; one isn't enough.
I'm not sure whether it's a good idea or not to call it a className;
it seems like a somewhat reasonable association with an existing
feature, but it might lead to confusion because of the ways it's not
analogous to that feature.

(We might also need a namespace here that's reserved to the
implementation and then a namespace for authors to use.)


(The original thing that prompted this, for what it's worth, was
type-ahead find in the browser, where the original idea was to
highlight the current find instance one way (we do that) and all the
others another way (we've never gotten to doing that). We also want
one for spellcheck, two (I think) for IME, one for the regular
browser selection, and probably others.)

> For regular content, I think we
> want a pseudo-element similar to ::selection <
> http://www.w3.org/TR/2005/WD-css3-selectors-20051215/> (note that this has
> been removed from the CSS3 Selectors TR, but I'm not aware of the details
> on why that is) which matches multiple marked ranges of a document. We can
> restrict the set of CSS properties honored by this pseudo-element to color,
> background, text-decoration, and outline (plus perhaps other useful
> properties).
>
> The MarkableDocument interface will then look like below:
>
> [Supplemental=Document]
> interface MarkableDocument {
> // this range should be a snapshot. The modification won't update the
> marker.
> void addMarker(Range range);
> void removeMarker(Range marker);
> sequence<Range> getMarkers();
> };

I rather dislike the use of the term "marker" here. I think using
mark as the verb would make more sense, perhaps as in:

[Supplemental=Document]
interface MarkableDocument {
void markRange(Range range, DOMString className);
void unmarkRange(Range range, DOMString className);
// I don't have strong opinions about what the API to access the
// current set of marked ranges looks like.

};


> And the CSS ::marked pseudo-element would match the markers associated with
> a document.

Makes sense, but with an argument for the name.

> For input and textarea, we need an entirely different solution, as their
> contents are not accessible using regular DOM tree operations. The
> proposed MarkableElement lacks support for multiple misspelling ranges, so
> we would need something better. Here is my proposal:

Makes sense, modulo keeping it analogous to any changes to the
document interface.

> The MarkedSection interface is semantically compatible with the existing
> selectionStart/selectionEnd properties of these elements. In order to
> style the marked sections, we would need a pseudo-class (named :marked or
> something similar) which would only apply to input and textarea elements,
> and which would also honor the same properties as the ::marked
> pseudo-element.

To style the ranges inside the form control, you still need a
pseudo-element. And I think the same pseudo-element is sufficient.

(Or are you talking about styling the form control that contains
ranges? In that case, yes, it would need to be a pseudo-class.)

[Ehsan Akhgari]

If we want to allow multiple ranges to be associated with the same named
category, that would require the UA to support multi-range selections,
right?

--
Ehsan
<http://ehsanakhgari.org/>

[L. David Baron]

On Tuesday 2011-11-29 16:33 -0500, Ehsan Akhgari wrote:
> If we want to allow multiple ranges to be associated with the same named
> category, that would require the UA to support multi-range selections,
> right?

I'm not sure what you mean by "support multi-range selections". It
certainly doesn't require that the browser expose UI for causing
multiple ranges to be part of the selection (i.e., the selection as
opposed to the IME or the spellcheck or any of the other
selection-like things), as Gecko currently does. It also doesn't
require that the browser even programmatically allow the selection
(again, as opposed to these other things) to consist of multiple
ranges.

But I certainly one to be able to highlight more than one misspelled
word at once, or more than one occurrence of a search at once. So I
definitely want to be able to support the concept of styling a named
set-of-ranges.

[Ryosuke Niwa]

On Tue, Nov 29, 2011 at 1:33 PM, Ehsan Akhgari <ehsan....@gmail.com>wrote:
>
> If we want to allow multiple ranges to be associated with the same named
> category, that would require the UA to support multi-range selections,
> right?
>

That might be one way to implement this but the API proposed here doesn't
impose such implementation on UAs. FWIW, WebKit has "markers" separate from
selection.

I guess one requirement for us is that marker shouldn't require any
re-layout. So styles like underline, line-through, color, background-color
etc... are fine but font-weight, line-height, etc... are not.

- Ryosuke

[Andreas Gal]

Any objections to split spell checking and the marker API? The former seems straight forward and ready to implement, the latter we can iterate over a bit more until there is consensus.

Andreas

[Ryosuke Niwa]

Splitting the two sounds good to me especially because the marker API
appears to be useful even outside of spellchecking.

- Ryosuke

[Hajime Morrita]

I'm sorry for my late jumping in.
Anyway, +1 for splitting.
--
morrita

--
morrita

[Hajime Morrita]

On Wed, Nov 30, 2011 at 6:48 AM, Ryosuke Niwa <rn...@webkit.org> wrote:
> On Tue, Nov 29, 2011 at 1:33 PM, Ehsan Akhgari <ehsan....@gmail.com>
> wrote:
>>
>> If we want to allow multiple ranges to be associated with the same named
>> category, that would require the UA to support multi-range selections,
>> right?
>
> That might be one way to implement this but the API proposed here doesn't
> impose such implementation on UAs. FWIW, WebKit has "markers" separate from
> selection.

Yes, WebKit can manage multiple markers
and it's separate concept from the selection (that WebKit can hold at most one).
So I guess we don't need MarkedSection for supporting multiple marked ranges.
I agree that virtual node concept possibly complicate the situation.
But I don't want to have yet another range-like stuff.

I noticed that "markableText" is too specific name though.
It can have more generic name like "valueNode" or "valueText"
to only imply the relationship with the "value" attribute.
Then this can be shared with future composition/IME feature.

Ehsan, what do you think?
I'm not familiar with Gecko internal and
may rely on unintentional assumption based on WebKit behavior.

By the way, I support mark/unmark verb naming.
It sounds making sense.

--
morrita

[Hajime Morrita]

A bit off-topic but I just remembered
what happened on our internal discussion before...

For spelling API, the only tricky bit is about privacy.
Users' "learned" words can be thought as a part of their personal information,
So revealing it to the page without any notification is a possible
privacy leakage.

There are some options to address this:

A: Use only anonymous (default) dictionary for the API.
B: Ask user's permission by the way some other API (ex. Filesystem
API) is doing.
C: Combination of A+B
D: Assuming there is no practical abuse for the API, just allow it.

Honestly I'm tempted to pick D.
But expecting that the discussion about privacy will happen in public anyway
and we should be conservative on this kind of topic,
C looks reasonable for me. It's flexible and extensible enough.
(And for some platforms including iOS, it would be difficult to
support A-only option because,
for example, the iOS UITextChecker provides no anonymous dictionary access.)

The permission based API can look like this:

[Callback]
interface SpellCheckHolderCallback {
void handle(SpellChecker spelling);
};

[Supplemental=Navigator]
interface SpellCheckerHodler {
const String SPELLCHECK_ANONYMOUS = ...; // will return without any
permission.
const String SPELLCHECK_USER = ...; // will ask permission.
void requestSpellChecker(SpellCheckHolderCallback, String type);
};

I'm sorry that I didn't mention this earlier.
But I'd like to hear your opinion.

--
morrita

On Wed, Nov 30, 2011 at 6:54 AM, Andreas Gal <g...@mozilla.com> wrote:
>
> Any objections to split spell checking and the marker API? The former seems
> straight forward and ready to implement, the latter we can iterate over a
> bit more until there is consensus.
> Andreas

> On Nov 29, 2011, at 10:48 PM, Ryosuke Niwa wrote:
>
> On Tue, Nov 29, 2011 at 1:33 PM, Ehsan Akhgari <ehsan....@gmail.com>
> wrote:
>>
>> If we want to allow multiple ranges to be associated with the same named
>> category, that would require the UA to support multi-range selections,
>> right?
>
> That might be one way to implement this but the API proposed here doesn't
> impose such implementation on UAs. FWIW, WebKit has "markers" separate from
> selection.

[Hajime Morrita]

[Lawrence Mandel]

> For spelling API, the only tricky bit is about privacy.
> Users' "learned" words can be thought as a part of their personal
> information,
> So revealing it to the page without any notification is a possible
> privacy leakage.

One specific issue that I've noticed in some cases is that a user dictionary may kick in and store a password. In this case it is also a security concern to expose a user dictionary to any page/app.

Lawrence

[Hironori Bono (坊野 博典)]

Greetings Andreas,

Thanks for conducting this discussion.
I do not have any objections to it. It is a good option to start
implementing the spell-checking API now. :)

Regards,

Hironori Bono
E-mail: hb...@google.com

[Hironori Bono (坊野 博典)]

Greetings Ehsan,

Many thanks for your proposal.
Since I have posted my first marker-API proposal, I have talked with
JavaScript developers about the API and heard their feedback. Among
their feedback, the most notable one is: they would like to use Range
for adding a marker also to text in an <input> element or a <textarea>
elements. (Even though they convinced these elements stored text into
their value attributes, they asked for me to find possible solutions
that allow them to use Range.) As a possible solution for this
problem, I started thinking of adding an interface that returns a
'virtual' node to these elements as listed below.

[Supplemental=HTMLInputElement, HTMLTextareaElement]
interface MarkableElement {
// Returns a text node consisting of the text stored in the 'value'
attribute. The returned node is used only for add styles to the text.
Node getMarkerNode(); // This function name is not a good one. Any
alternatives are welcome.
};

To use this interface, we can add a marker as we add one for Markerble
documents as listed in the following snippet.

<body>
<textarea id="text1" rows="10" cols="80">Theer is a misspelled
word.</textarea>
<script type="text/javascript" language="javascript">
var node = document.getElementById('text1').getMarkerNode();
var range = docment.createRange();
range.setStart(node, 0);
range.setEnd(node, 5);
node.addMarker(range);
</script>
</body>

Even though this idea is an infant phase, I wish it helps.

Regards,

Hironori Bono
E-mail: hb...@google.com

On Wed, Nov 30, 2011 at 1:58 AM, Ehsan Akhgari <ehsan....@gmail.com> wrote:
> Hi all,
>
> I think SpellChecker is a great interface which is capable of handling both
> spelling and grammar checking, and the fact that it can be implemented
> asynchronously makes it a really good choice in face of different types of
> spelling back-ends.
>

> I'm not quite comfortable with the DocumentMarker interface, though.  I
> think that it's a bad idea to specify a className for a range, since
> classNames apply to DOM elements currently, and making them "apply" to

> ranges in this way would be confusing.  For regular content, I think we want

> a pseudo-element similar to ::selection
> <http://www.w3.org/TR/2005/WD-css3-selectors-20051215/> (note that this has
> been removed from the CSS3 Selectors TR, but I'm not aware of the details on
> why that is) which matches multiple marked ranges of a document.  We can
> restrict the set of CSS properties honored by this pseudo-element to color,
> background, text-decoration, and outline (plus perhaps other useful
> properties).
>
> The MarkableDocument interface will then look like below:
>
> [Supplemental=Document]
> interface MarkableDocument {
>   // this range should be a snapshot. The modification won't update the
> marker.
>   void addMarker(Range range);
>   void removeMarker(Range marker);
>   sequence<Range> getMarkers();
> };
>

> And the CSS ::marked pseudo-element would match the markers associated with
> a document.
>

> (Note that this proposal does not allow support for different marker types,
> so we might want to make this a bit less generic.)
>

> For input and textarea, we need an entirely different solution, as their
> contents are not accessible using regular DOM tree operations.  The proposed
> MarkableElement lacks support for multiple misspelling ranges, so we would
> need something better.  Here is my proposal:
>

> interface MarkedSection {
>   unsigned long start;
>   unsigned long end;
> };
>
> [Supplemental=HTMLInputElement, HTMLTextareaElement]
> interface MarkableElement {
>   void addMarker(MarkedSection section);
>   void removeMarker(MarkedSection section);
>   sequence<MarkedSection> getMarkers();
> };
>

> The MarkedSection interface is semantically compatible with the existing
> selectionStart/selectionEnd properties of these elements.  In order to style
> the marked sections, we would need a pseudo-class (named :marked or
> something similar) which would only apply to input and textarea elements,
> and which would also honor the same properties as the ::marked
> pseudo-element.
>

> I'd be interested to know what you all think about this proposal for the
> marking part.
>
>
> Cheers,
> --
> Ehsan
> <http://ehsanakhgari.org/>
>
>

> On Tue, Nov 29, 2011 at 2:49 AM, Andreas Gal <g...@mozilla.com> wrote:
>>
>>
>> Supporting grammar and spelling errors is definitely a good idea. I like
>> the new API. We can fine-tune it as we go along. We should file bugs for
>> webkit and Gecko to get an implementation going. I will find someone to own
>> this on our end.
>>
>> Adding dbaron and bz to discuss the marking/highlighting API. A separate
>> more generic API sounds like the way to go. david, bz, what do you think?
>>

>> Andreas

[Andreas Gal]

We discussed this a bit earlier. We can have per-domain private dictionaries, and if the user grants a special privilege the site gets access to the UA-wide private dictionary. That avoids privacy leaks. I don't think we need a programmable API for this. The UA can handle all this.

Andreas

On Nov 30, 2011, at 2:56 AM, Hajime Morrita wrote:

> A bit off-topic but I just remembered
> what happened on our internal discussion before...
>

> For spelling API, the only tricky bit is about privacy.
> Users' "learned" words can be thought as a part of their personal information,
> So revealing it to the page without any notification is a possible
> privacy leakage.
>

> On Wed, Nov 30, 2011 at 6:54 AM, Andreas Gal <g...@mozilla.com> wrote:
>>
>> Any objections to split spell checking and the marker API? The former seems
>> straight forward and ready to implement, the latter we can iterate over a
>> bit more until there is consensus.
>> Andreas
>> On Nov 29, 2011, at 10:48 PM, Ryosuke Niwa wrote:
>>
>> On Tue, Nov 29, 2011 at 1:33 PM, Ehsan Akhgari <ehsan....@gmail.com>
>> wrote:
>>>
>>> If we want to allow multiple ranges to be associated with the same named
>>> category, that would require the UA to support multi-range selections,
>>> right?
>>
>> That might be one way to implement this but the API proposed here doesn't
>> impose such implementation on UAs. FWIW, WebKit has "markers" separate from
>> selection.
>> I guess one requirement for us is that marker shouldn't require any
>> re-layout. So styles like underline, line-through, color, background-color
>> etc... are fine but font-weight, line-height, etc... are not.
>> - Ryosuke
>>
>>
>
>
>

> --
> morrita

[Hajime Morrita]

On Wed, Nov 30, 2011 at 2:32 PM, Andreas Gal <g...@mozilla.com> wrote:
>
> We discussed this a bit earlier. We can have per-domain private dictionaries, and if the user grants a special privilege the site gets access to the UA-wide private dictionary. That avoids privacy leaks. I don't think we need a programmable API for this. The UA can handle all this.

Ah, that sounds right.
Although the implementation may be started without supporting learning words,
hiding such details like dictionary feels like a good direction.

Thanks for your suggestion.
--
morrita

--
morrita

[Ehsan Akhgari]

On Tue, Nov 29, 2011 at 4:40 PM, L. David Baron <dba...@mozilla.com> wrote:

> On Tuesday 2011-11-29 16:33 -0500, Ehsan Akhgari wrote:
> > If we want to allow multiple ranges to be associated with the same named
> > category, that would require the UA to support multi-range selections,
> > right?
>

> I'm not sure what you mean by "support multi-range selections". It
> certainly doesn't require that the browser expose UI for causing
> multiple ranges to be part of the selection (i.e., the selection as
> opposed to the IME or the spellcheck or any of the other
> selection-like things), as Gecko currently does. It also doesn't
> require that the browser even programmatically allow the selection
> (again, as opposed to these other things) to consist of multiple
> ranges.
>
> But I certainly one to be able to highlight more than one misspelled
> word at once, or more than one occurrence of a search at once. So I
> definitely want to be able to support the concept of styling a named
> set-of-ranges.
>

Sorry, I believe <https://bugzilla.mozilla.org/show_bug.cgi?id=256773#c4>
was what threw me off here. I now agree that there is nothing inherent in
that proposal which requires support for multi-range selections.

[Ehsan Akhgari]

On Tue, Nov 29, 2011 at 8:02 PM, Hajime Morrita <mor...@google.com> wrote:

> On Wed, Nov 30, 2011 at 6:48 AM, Ryosuke Niwa <rn...@webkit.org> wrote:

> > On Tue, Nov 29, 2011 at 1:33 PM, Ehsan Akhgari <ehsan....@gmail.com>

> > wrote:
> >>
> >> If we want to allow multiple ranges to be associated with the same named
> >> category, that would require the UA to support multi-range selections,
> >> right?
> >

> > That might be one way to implement this but the API proposed here doesn't
> > impose such implementation on UAs. FWIW, WebKit has "markers" separate
> from
> > selection.
>

> Yes, WebKit can manage multiple markers
> and it's separate concept from the selection (that WebKit can hold at most
> one).
> So I guess we don't need MarkedSection for supporting multiple marked
> ranges.
> I agree that virtual node concept possibly complicate the situation.
> But I don't want to have yet another range-like stuff.
>

The only reason why we would need MarkedSection is for inputs and textareas
which do not expose their inner DOM. Do you have another idea on how we
would handle that?

Note that my proposal does _not_ used MarkedSection for anything else.

> I noticed that "markableText" is too specific name though.
> It can have more generic name like "valueNode" or "valueText"
> to only imply the relationship with the "value" attribute.
> Then this can be shared with future composition/IME feature.
>

I'm ambivalent on the naming. :-)

> Ehsan, what do you think?
> I'm not familiar with Gecko internal and
> may rely on unintentional assumption based on WebKit behavior.
>

Gecko currently uses selections internally for misspell highlighting, but
that's totally an implementation detail and should not get in the way of
the new API.

[Ehsan Akhgari]

2011/11/30 Hironori Bono (坊野 博典) <hb...@google.com>

I think doing this is opening a can of worms. :(

By doing that, we need to make sure that operations on this node would make
sense. We also need to ensure that passing this node to anything which
accepts a node results in something sane. We also need to special case
ranges to check for these node types, and ensure that a range which holds
one end under such a node has its other end on the same input/textarea
element. Also, doing this means that we need to expose addMarker on nodes,
as opposed to documents, which doesn't make sense for any other node type.
I *really* don't want to go through all this pain...

I totally sympathize with web developers wanting this. The way that the
input and textarea APIs have emerged on the web totally sucks, and this is
only one of the reasons why. But this idea seems more like a spot fix to
me as opposed to a general improvement for input/textarea.

--
Ehsan
<http://ehsanakhgari.org/>

[Ehsan Akhgari]

There's the issue of fingerprinting by allowing any user dictionary to be
included in this API (whether it's domain-wide or not). Since the API
doesn't allow enumerating all of the words in the user dictionary, the
fingerprinting might be a bit hard to perform, but still this will totally
fail in the fact of dictionary attacks.

Sid is an expert in this area, so I will defer to him on which solution is
more acceptable from a privacy standpoint. But I think we can focus on
ignoring user dictionaries at the initial stages of implementation at least.

Cheers,
--
Ehsan
<http://ehsanakhgari.org/>

> >>> On Nov 29, 2011, at 10:48 PM, Ryosuke Niwa wrote:
> >>>
> >>> On Tue, Nov 29, 2011 at 1:33 PM, Ehsan Akhgari <
> ehsan....@gmail.com>
> >>> wrote:
> >>>>
> >>>> If we want to allow multiple ranges to be associated with the same
> named
> >>>> category, that would require the UA to support multi-range selections,
> >>>> right?
> >>>
> >>> That might be one way to implement this but the API proposed here
> doesn't
> >>> impose such implementation on UAs. FWIW, WebKit has "markers" separate
> from
> >>> selection.

[Sid Stamm]

Hey Ehsan,

For what it's worth, I agree with the statement made by Andreas below:

> We discussed this a bit earlier. We can have per-domain private
> dictionaries, and if the user grants a special privilege the site
> gets access to the UA-wide private dictionary. That avoids privacy
> leaks. I don't think we need a programmable API for this. The UA
> can handle all this.

Seems reasonable to me to require the UA to handle dictionary choice,
which lets each UA decide how strict they want to be with custom
dictionaries vs. fingerprinting. I would advocate per-domain private
dictionaries as a good direction, so long as dictionaries are only
deployed at a top level document, or at least one where the user has
interacted with text; if one can load and query a dictionary invisibly
without user interaction, an invisible iframe could be used as a
tracking device. :)

-Sid

> <ehsan....@gmail.com <mailto:ehsan....@gmail.com>>

[Ehsan Akhgari]

FWIW, the proposed API can be used without user interaction.

Cheers,
--
Ehsan
<http://ehsanakhgari.org/>

[Hajime Morrita]

On Thu, Dec 1, 2011 at 1:07 AM, Ehsan Akhgari <ehsan....@gmail.com> wrote:
> On Tue, Nov 29, 2011 at 8:02 PM, Hajime Morrita <mor...@google.com> wrote:
>>

>> On Wed, Nov 30, 2011 at 6:48 AM, Ryosuke Niwa <rn...@webkit.org> wrote:
>> > On Tue, Nov 29, 2011 at 1:33 PM, Ehsan Akhgari <ehsan....@gmail.com>

>> > wrote:
>> >>
>> >> If we want to allow multiple ranges to be associated with the same
>> >> named
>> >> category, that would require the UA to support multi-range selections,
>> >> right?
>> >
>> > That might be one way to implement this but the API proposed here
>> > doesn't
>> > impose such implementation on UAs. FWIW, WebKit has "markers" separate
>> > from
>> > selection.
>>

>> Yes, WebKit can manage multiple markers
>> and it's separate concept from the selection (that WebKit can hold at most
>> one).
>> So I guess we don't need MarkedSection for supporting multiple marked
>> ranges.
>> I agree that virtual node concept possibly complicate the situation.
>> But I don't want to have yet another range-like stuff.
>
> The only reason why we would need MarkedSection is for inputs and textareas
> which do not expose their inner DOM.  Do you have another idea on how we
> would handle that?

My intention was that "valueText" is not an actual inner (shadow) DOM
but a proxy for, or snapshot of, the text string of the field.
I choose Text instead of String just because it allows us to reuse
Range for specifying a section.

On the other hand, using Range can feed its own worm as you mentioned
in spelling API.
So now I agree that using MarkedSection will be a reasonable choice.

By the way, the current API is adding a certain number of method to
existing objects
and I personally feel it like a namespace pollution.

How about this instead?

----

interface MarkedRegion {
// TBD: A way to identifying the marker type is just a placeholder:
We need to discuss this separately.
readonly attribute DOMString name;
};

interface MarkedRange extends MarkedRegion {
readonly attribute Range range;
}

interface MarkedSection extends MarkedRegion {
readonly unsigned long start;
readonly unsigned long end;
};

interface DocumentMarker {
// TBD: A way to giving the marker type is just a placeholder: We
need more discussion.
void mark(Range range, DOMString name);
void mark(HTMLInputElement element, unsigned long start, unsigned
long end, DOMString name);
void mark(HTMLTextareaElement element, unsigned long start, unsigned
long end, DOMString name);

// Unmarking single region.
void unmark(MarkedRegion region);
// Allowing to unmark multiple regions at once.
void unmark(Range range, [Optional] DOMString name);
void unmark(HTMLInputElement element, [Optional] DOMString name);
void unmark(HTMLTextareaElement element, [Optional] DOMString name);
// Unmarking by name is usable for clearing highights at once.
WebKit uses similar API internally.
void unmark(DOMStrin name);

sequence<MarkedRegion> find(Range range, [Optional] DOMString name);
sequence<MarkedRegion> find(HTMLInputElement element, [Optional]
DOMString name);
sequence<MarkedRegion> find(HTMLTextareaElement element, [Optional]
DOMString name);
};

[Supplemental=Document]
interface MarkableDocument {
readonly attribute DocumentMarker marker;
};

----

This extracted a set of new API into a single entry object called
DocumentMarker.
This will make it easier to extend/change without worrying about
possible name conflicts.
This proposal also contains MarkedRegion super-type which provides a
unified access to the marked range/section.
Web developer can check the subtype of the region by coding like "if
(markedRegion.range) { ... }".

Maybe this is a premature thought and we can just extend Document interface.
Ehsan, what do you think about this?

--
morrita

>
> Note that my proposal does _not_ used MarkedSection for anything else.
>
>>
>> I noticed that "markableText" is too specific name though.
>> It can have more generic name like "valueNode" or "valueText"
>> to only imply the relationship with the "value" attribute.
>> Then this can be shared with future composition/IME feature.
>
> I'm ambivalent on the naming.  :-)
>
>>
>> Ehsan, what do you think?
>> I'm not familiar with Gecko internal and
>> may rely on unintentional assumption based on WebKit behavior.
>
> Gecko currently uses selections internally for misspell highlighting, but
> that's totally an implementation detail and should not get in the way of the
> new API.
>

> Cheers,
> --
> Ehsan
> <http://ehsanakhgari.org/>

--
morrita

[Hironori Bono (坊野 博典)]

Greetings Ehsan,

It is definitely not my intention to open such a can and start a
long-lasting discussion. :)
In my honest opinion, your proposal is good for me. Nevertheless, it
may be better for us to add some more description to clarify the
behavior of the start and length parameters of MarkerSection because I
have received some questions about them, such as should we count a
combined character (or a surrogate pair) as one character?

Regards,

Hironori Bono
E-mail: hb...@google.com

[Hajime Morrita]

On Wed, Nov 30, 2011 at 5:51 AM, L. David Baron <dba...@mozilla.com> wrote:
> On Tuesday 2011-11-29 11:58 -0500, Ehsan Akhgari wrote:

>> I'm not quite comfortable with the DocumentMarker interface, though.  I
>> think that it's a bad idea to specify a className for a range, since
>> classNames apply to DOM elements currently, and making them "apply" to
>> ranges in this way would be confusing.
>

> I think we do need multiple names for ranges here; one isn't enough.
> I'm not sure whether it's a good idea or not to call it a className;
> it seems like a somewhat reasonable association with an existing
> feature, but it might lead to confusion because of the ways it's not
> analogous to that feature.
>
> (We might also need a namespace here that's reserved to the
> implementation and then a namespace for authors to use.)
>
>
> (The original thing that prompted this, for what it's worth, was
> type-ahead find in the browser, where the original idea was to
> highlight the current find instance one way (we do that) and all the
> others another way (we've never gotten to doing that).  We also want
> one for spellcheck, two (I think) for IME, one for the regular
> browser selection, and probably others.)
>

>> For regular content, I think wet

>> want a pseudo-element similar to ::selection <
>> http://www.w3.org/TR/2005/WD-css3-selectors-20051215/> (note that this has
>> been removed from the CSS3 Selectors TR, but I'm not aware of the details
>> on why that is) which matches multiple marked ranges of a document.  We can
>> restrict the set of CSS properties honored by this pseudo-element to color,
>> background, text-decoration, and outline (plus perhaps other useful
>> properties).
>>
>> The MarkableDocument interface will then look like below:
>>
>> [Supplemental=Document]
>> interface MarkableDocument {
>>   // this range should be a snapshot. The modification won't update the
>> marker.
>>   void addMarker(Range range);
>>   void removeMarker(Range marker);
>>   sequence<Range> getMarkers();
>> };
>

> I rather dislike the use of the term "marker" here.  I think using
> mark as the verb would make more sense, perhaps as in:
>
> [Supplemental=Document]
> interface MarkableDocument {
>  void markRange(Range range, DOMString className);
>  void unmarkRange(Range range, DOMString className);
>  // I don't have strong opinions about what the API to access the
>  // current set of marked ranges looks like.
> };
>
>

>> And the CSS ::marked pseudo-element would match the markers associated with
>> a document.
>

> Makes sense, but with an argument for the name.
>

I'm +1 for this.

So for the Marker API, what we need in CSS space are basically:

- A. A way to select a set of named markers. (The name is the one
given through Marker API.)
- B. A way to customize the visual representation (or style) of named
markers and existing marker-like features (spellchecking, in-page
search highlight, etc.)
- C. A way to give spellcheck-like visual representation (wiggle line)
to named markers.


For A, we can just adopt David's proposal:
Introducing a pseudo element called "marked", which accepts one
argument for the name of a marker group.

:marked(myhilight) { .... }

Also, we can define additional set of pseudo elements for built-in
markers. (The names are tentative.)

- :marked-spelling
- :marked-grammar
- :marked-search-highlight
- :marked-correction-indicator

Having separate pseudo elements for built-ins instead of just using marked(),
we can avoid namespace problem.


For B,

- B1: we can allow limited set of properties for giving styles of
marker's pseudo elements. Or,
- B2: we can introduce new "marker" property: It's something like
"marker: <underline-type>, <underline-color>, <foreground-color>
,<background-color>",
which is the only way to give the style of markers.

Using B1, we can reuse existing vocabulary.
At the same time, we need to carefully limit which property can be
allowed for markers
because, as Ryosuke mentioned, these properties shouldn't affect the
page layout.

I'm a bit worry about that this whitelist of allowed properties is
rather arbitrary and
we can easily fall into bikeshed while discussing this publicly.
What worse can happen: we possibly need to explain the marker concept
in terms of the box model
to make it an unified part of CSS rendering model. That sounds like a nightmare.

Using B2, we need to invent new vocabulary for markers.
Also it's inflexible. But I think such inflexibility reflects the
state of our marker implementation.
At least in WebKit, marker rendering is just a small twist in the painting path.
And what we need is (in my understanding) a way to customize some
variables which are used in this twist,
not the extended box model which supports yet another way to give
style properties.

If the whitelist of B1 is clear and trivial enough, I think B1 is
desirable. Otherwise, B2 is preferable for me.


For C, assuming B2, we can define some special values for marker
property like "marker: spellchecking" and so on.
If we go B1, we can extend text-decoration, maybe.

My proposal here may be too rough.
I just hope this helps to move the discussion and love to hear your opinion.

--
morrita

>> For input and textarea, we need an entirely different solution, as their
>> contents are not accessible using regular DOM tree operations.  The
>> proposed MarkableElement lacks support for multiple misspelling ranges, so
>> we would need something better.  Here is my proposal:
>

> Makes sense, modulo keeping it analogous to any changes to the
> document interface.
>

>> The MarkedSection interface is semantically compatible with the existing
>> selectionStart/selectionEnd properties of these elements.  In order to
>> style the marked sections, we would need a pseudo-class (named :marked or
>> something similar) which would only apply to input and textarea elements,
>> and which would also honor the same properties as the ::marked
>> pseudo-element.
>

> To style the ranges inside the form control, you still need a
> pseudo-element.  And I think the same pseudo-element is sufficient.
>
> (Or are you talking about styling the form control that contains
> ranges?  In that case, yes, it would need to be a pseudo-class.)
>

> -David
>
> --
> 𝄞   L. David Baron                         http://dbaron.org/   𝄂
> 𝄢   Mozilla                           http://www.mozilla.org/   𝄂
>

--
morrita

[Gervase Markham]

On 30/11/11 05:32, Andreas Gal wrote:
>
> We discussed this a bit earlier. We can have per-domain private
> dictionaries, and if the user grants a special privilege the site
> gets access to the UA-wide private dictionary. That avoids privacy
> leaks. I don't think we need a programmable API for this. The UA can
> handle all this.

That sounds to me a bit like making the user model fit the security
model, and not the other way around.

"Gerv" is not in any dictionaries. Am I really going to have to click
"add to dictionary" for every site I do spell-checking on? Or have a
mental concept of the idea that I don't just have one personal
dictionary, but a large number of them? Or give every site permission
"yes, you can use my dictionary"?

Could someone quantify the privacy risk? What words might I add to my
user dictionary which, when found via a random search, would reveal
something about myself that I'd rather keep private?

Gerv

[Gervase Markham]

On 30/11/11 05:32, Andreas Gal wrote:
>

> We discussed this a bit earlier. We can have per-domain private
> dictionaries, and if the user grants a special privilege the site
> gets access to the UA-wide private dictionary. That avoids privacy
> leaks. I don't think we need a programmable API for this. The UA can
> handle all this.

[Gervase Markham]

On 30/11/11 05:32, Andreas Gal wrote:
>

> We discussed this a bit earlier. We can have per-domain private
> dictionaries, and if the user grants a special privilege the site
> gets access to the UA-wide private dictionary. That avoids privacy
> leaks. I don't think we need a programmable API for this. The UA can
> handle all this.

[Ryosuke Niwa]

On Thu, Dec 1, 2011 at 4:20 AM, Gervase Markham <ge...@mozilla.org> wrote:
>
> That sounds to me a bit like making the user model fit the security model,
> and not the other way around.
>
> "Gerv" is not in any dictionaries. Am I really going to have to click "add
> to dictionary" for every site I do spell-checking on? Or have a mental
> concept of the idea that I don't just have one personal dictionary, but a
> large number of them? Or give every site permission "yes, you can use my
> dictionary"?
>
> Could someone quantify the privacy risk? What words might I add to my user
> dictionary which, when found via a random search, would reveal something
> about myself that I'd rather keep private?
>

It could be your real name, or even password if words in the dictionary are
automatically added by some software. It may also signal that the client
has a certain software installed with known exploits.

Also, revealing which words are in the dictionary will make fingerprinting
a user much easier.

- Ryosuke

[Gervase Markham]

On 01/12/11 20:19, Ryosuke Niwa wrote:
> It could be your real name,

Having a word in a dictionary doesn't mean it's my real name.

> or even password if words in the dictionary are
> automatically added by some software.

Well, that would be rather dumb software!

> It may also signal that the client
> has a certain software installed with known exploits.

That seems fairly far-fetched to me. And normally, skript kiddies don't
bother testing for software versions, they just attempt an exploit. If
it fails, the software's the wrong version :-)

> Also, revealing which words are in the dictionary will make fingerprinting
> a user much easier.

Now _that_ is perhaps a better reason :-|

We need to define what the win condition is for fingerprinting. We've
been trying to improve things for passive fingerprinting (stuff a site
can find out without running JS code), but I'm fairly sure that making
browsers unfingerprintable actively (when the site runs code) is a very
difficult goal. There are all sorts of things such as supported fonts
which vary from browser to browser; we aren't going to remove all those
APIs.

Gerv

[Hironori Bono (坊野 博典)]

Greetings Andreas,

Have you filed a bug about this spell-checking API to mozilla.org, or
uploaded its reference document? They are definitely helpful for us to
file a WebKit bug.
Thank you for your work in advance.

Regards,

Hironori Bono
E-mail: hb...@google.com

On Wed, Nov 30, 2011 at 1:10 PM, Hironori Bono (坊野 博典) <hb...@google.com> wrote:
> Greetings Andreas,
>
> Thanks for conducting this discussion.
> I do not have any objections to it. It is a good option to start
> implementing the spell-checking API now. :)
>

> Regards,
>
> Hironori Bono
> E-mail: hb...@google.com
>

> On Wed, Nov 30, 2011 at 6:54 AM, Andreas Gal <g...@mozilla.com> wrote:
>>
>> Any objections to split spell checking and the marker API? The former seems
>> straight forward and ready to implement, the latter we can iterate over a
>> bit more until there is consensus.
>>
>> Andreas
>>

>> On Nov 29, 2011, at 10:48 PM, Ryosuke Niwa wrote:
>>
>> On Tue, Nov 29, 2011 at 1:33 PM, Ehsan Akhgari <ehsan....@gmail.com>
>> wrote:
>>>
>>> If we want to allow multiple ranges to be associated with the same named
>>> category, that would require the UA to support multi-range selections,
>>> right?
>>
>>
>> That might be one way to implement this but the API proposed here doesn't
>> impose such implementation on UAs. FWIW, WebKit has "markers" separate from
>> selection.
>>

[Mounir Lamouri]

On 11/28/2011 01:13 PM, Andreas Gal wrote:
> navigator.language is the default, but we should probably add an
> optional language parameter so its possible to spellcheck for a
> different language than the default

I think we should use the document's language [1] by default instead of
UA's language (navigator.language) and fallback to UA's language if
there is no matching dictionary available.
If I speak English, French, German and Japanese but my UA is in English,
I might still have four dictionaries and will be very interested to have
this API using the French one if I happen to write a comment on a French
website.

I think we should also allow the language to be specified.
Maybe even allowing an element to be passed. That might be useful to say
"use that element's language" which is unfortunately harder than doing
element.lang because the language of an element is element.lang if
specified or the the closest parent's language. Though, this is assuming
script might not know already the element's language. That might be the
case if the content is generated and localized. If that seems unusual
enough, that could wait for another iteration of the specification.
Actually, in the use case of an IME, passing the element to the method
would be a great help I believe.

> Words can be added to/removed from the personal dictionary:
>
> navigator.spelling.recognize("word")
> navigator.spelling.forget("word")
>
> The personal dictionary can be queried with:
>
> navigator.spelling.enumerate(callback)

What would be the uses cases for this for a website without any specific
permission? Why would a website want me to recognize a specific set of
words only when I am in this website?

I agree that a website with some privileges might want to add and remove
words from the user personal dictionaries. Though, I don't see the point
with allowing iterating those. Also, I don't think we should try to add
privileged-only parts in the API for the first iteration.

On 11/30/2011 02:56 AM, Hajime Morrita wrote:
> For spelling API, the only tricky bit is about privacy.
> Users' "learned" words can be thought as a part of their personal information,
> So revealing it to the page without any notification is a possible
> privacy leakage.
>
> There are some options to address this:
>
> A: Use only anonymous (default) dictionary for the API.
> B: Ask user's permission by the way some other API (ex. Filesystem
> API) is doing.
> C: Combination of A+B
> D: Assuming there is no practical abuse for the API, just allow it.

I would like to add that being able to check even simple words is
already a privacy concern because it lets the website know about two things:
1. Does the user have a dictionary;
2. What are the available dictionaries languages.

Those issues might be considered as small; I even doubt a user can
download a release of a popular browser without a pre-installed
dictionary. However, I guess that makes having no dictionary a highly
sensitive privacy bit.
Also, your dictionaries and their languages might become an efficient
way to fingerprint you even if the website has no access to your
personal dictionary. People having more than one dictionary are unusual
enough (and the combination large enough) to be easily traceable with
that information.

I don't see any non-invasive or two restrictive way to prevent (1). I
think we could hope that UA will allow this API to be disabled so users
with no dictionary and cautious regarding their privacy will be able to
protect themselves.
Iterating trough dictionaries could be (nearly) prevented if we require
a user interaction to be able to access the API and forbid changing the
language without another interaction. This is not entirely fixing the
privacy issue but should highly reduce the abuses.

> Honestly I'm tempted to pick D.
> But expecting that the discussion about privacy will happen in public anyway
> and we should be conservative on this kind of topic,

This discussion is already public ;)

> C looks reasonable for me. It's flexible and extensible enough.
> (And for some platforms including iOS, it would be difficult to
> support A-only option because,
> for example, the iOS UITextChecker provides no anonymous dictionary access.)

I see two solutions for the personal dictionary issue:
We could leave this use case for a next iteration of the specification
and highly discourage/forbid its usage in the first iteration. In the
next iteration, we could add a method requesting privileges to access
personal dictionaries in navigator.spellcheck or use a DAP Permissions
API-like API to request those privileges [2].
The alternative would be to forbid having more than one call to the API
per user interaction which would make fingerprinting using user
dictionary seriously complex unless there a way to do a highly valuable
fingerprinting with only one (or a few words).

> The permission based API can look like this:
>
> [Callback]
> interface SpellCheckHolderCallback {
> void handle(SpellChecker spelling);
> };
>
> [Supplemental=Navigator]
> interface SpellCheckerHodler {
> const String SPELLCHECK_ANONYMOUS = ...; // will return without any
> permission.
> const String SPELLCHECK_USER = ...; // will ask permission.
> void requestSpellChecker(SpellCheckHolderCallback, String type);
> };

I would suggest the following API:

interface NavigatorSpelling {
readonly atribute SpellingManager; spelling;
}

Navigator implements NavigatorSpelling;

interface SpellingManager {
SpellCheckRequest check(DOMString word, [optional] DOMString lang);
SpellCheckRequest check(DOMString word, [optional] HTMLElement element);

SpellSuggestRequest suggest(DOMString word, [optional] DOMString lang);
SpellSuggestRequest suggest(DOMString word, [optional] HTMLElement
element);
};

interface SpellCheckRequest : EventTarget {
// readyState can be "processing" or "done".
readonly attribute DOMString readyState;
readonly attribute DOMError? error;
attribute EventListener onsuccess;
attribute EventListener onerror;
attribute readonly boolean? result;
};

interface SpellSuggestRequest : EventTarget {
// readyState can be "processing" or "done".
readonly attribute DOMString readyState;
readonly attribute DOMError? error;
attribute EventListener onsuccess;
attribute EventListener onerror;
attribute readonly DOMString[]? result;
};

Examples:

var r = navigator.spelling.check("this");
r.onsuccess = function() { alert("Spelling correctness is " + r.result); };
r.onerror = function() { alert("The following error occured: " +
r.error.name); };

var r = navigator.spelling.suggest("th");
r.onsuccess = function() { alert("Suggestions are: " + r.result); };
r.onerror = function() { alert("The following error occured: " +
r.error.name); };

Error handling would be able to handle the following situations:
- various errors: we are doing I/O and using third party libraries so
errors can always occur;
- no dictionary (we shouldn't show it as "NoDictionary" for privacy
reasons but show it as "various" I guess);
- unauthorized call (if the proposal of allowing only one call per user
interaction is accepted).

I believe most use cases are going to be like the following:
var r = navigator.spelling.check("something");
r.onsuccess = r.onerror = function() {
if (!r.result) {
// probably log the error
return;
}
// do something with the result
};

Feedbacks would be very welcome :)

[1] html.lang
[2] http://www.w3.org/TR/2010/WD-api-perms-20101005/

Thanks,
--
Mounir

[Robert Kaiser]

Mounir Lamouri schrieb:

> Those issues might be considered as small; I even doubt a user can
> download a release of a popular browser without a pre-installed
> dictionary.

Actually, e.g. German Firefox comes without a dictionary, due to the
only available German dictionaries (LGPL-only) not being available under
the license we use for shipping Firefox (MPL).

The same is also true for a number of other languages.

Robert Kaiser

[Ehsan Akhgari]

(Sorry that I lost track of this thread...)

On Thu, Dec 1, 2011 at 2:03 AM, Hajime Morrita <mor...@google.com> wrote:

> So for the Marker API, what we need in CSS space are basically:
>
> - A. A way to select a set of named markers. (The name is the one
> given through Marker API.)
> - B. A way to customize the visual representation (or style) of named
> markers and existing marker-like features (spellchecking, in-page
> search highlight, etc.)
> - C. A way to give spellcheck-like visual representation (wiggle line)
> to named markers.
>
>
> For A, we can just adopt David's proposal:
> Introducing a pseudo element called "marked", which accepts one
> argument for the name of a marker group.
>
> :marked(myhilight) { .... }
>
> Also, we can define additional set of pseudo elements for built-in
> markers. (The names are tentative.)
>

I think that is a good idea.

> - :marked-spelling
> - :marked-grammar
> - :marked-search-highlight
> - :marked-correction-indicator
>
> Having separate pseudo elements for built-ins instead of just using
> marked(),
> we can avoid namespace problem.
>

Hmm, I wonder if this is going to be an open-ended list... What is the
downside of predefining some known names for :marked, and let the author
change the way that find results are displayed the same way that they
customize the visual representation of every other marked range?

> For B,
>
> - B1: we can allow limited set of properties for giving styles of
> marker's pseudo elements. Or,
> - B2: we can introduce new "marker" property: It's something like
> "marker: <underline-type>, <underline-color>, <foreground-color>
> ,<background-color>",
> which is the only way to give the style of markers.
>
> Using B1, we can reuse existing vocabulary.
> At the same time, we need to carefully limit which property can be
> allowed for markers
> because, as Ryosuke mentioned, these properties shouldn't affect the
> page layout.
>
> I'm a bit worry about that this whitelist of allowed properties is
> rather arbitrary and
> we can easily fall into bikeshed while discussing this publicly.
> What worse can happen: we possibly need to explain the marker concept
> in terms of the box model
> to make it an unified part of CSS rendering model. That sounds like a
> nightmare.
>

The criterion for whether a property is allowed or not is whether it
changes the layout computations or not. So I don't think that we're going
to get into too much bikeshedding.

> Using B2, we need to invent new vocabulary for markers.
> Also it's inflexible. But I think such inflexibility reflects the
> state of our marker implementation.
> At least in WebKit, marker rendering is just a small twist in the painting
> path.
> And what we need is (in my understanding) a way to customize some
> variables which are used in this twist,
> not the extended box model which supports yet another way to give
> style properties.
>

I don't like the idea of putting everything in the marker property syntax,
since we would need to extend it every time that we decide that property X
would also be nice to support.

> If the whitelist of B1 is clear and trivial enough, I think B1 is
> desirable. Otherwise, B2 is preferable for me.
>

I prefer B1 very much.

> For C, assuming B2, we can define some special values for marker
> property like "marker: spellchecking" and so on.
> If we go B1, we can extend text-decoration, maybe.
>

Hmm, if we have B1 and also a new value for the text-decoration property
which draws wiggly lines, why would we need to specify anything extra for
this use case?

Cheers,
Ehsan