Replacing default __iterator__?

[Nickolay Ponomarev]

Hi,

I'm trying to replace the __iterator__ of an object with a custom
iterator, then iterate over the object in a for..in loop. It seems
that the loop only iterates over the intersection of two sets:
1) the set of properties that would be returned by the default iterator
2) the set of properties returned by the custom iterator

Here's the code I'm testing:
> var obj = {a:1, __iterator__: function() { return (function() { yield "a"; yield 2 })() } }
> for(var i in obj) print(i + "->" + obj[i])
a->1
> obj[2] = 4
4
> for(var i in obj) print(i + "->" + obj[i])
a->1
2->4

Is this behavior expected? I was confused by this and so were the
authors of this test:
http://lxr.mozilla.org/mozilla/source/js/tests/js1_7/iterable/basic-for-each.js
and this comment:
http://developer.mozilla.org/en/docs/Talk:New_in_JavaScript_1.7#Syntax_for_replacing_native_iterators.3F

AFAIK, Python doesn't allow to override __iter__ for dicts, which are
rather similar to JS objects, but it does allow to define a custom
__iter__ method on a class.

Nickolay

[Jeff Walden]

Nickolay Ponomarev wrote:
> Is this behavior expected? I was confused by this and so were the
> authors of this test:
> http://lxr.mozilla.org/mozilla/source/js/tests/js1_7/iterable/basic-for-each.js

I was? ;-) I admit to finding __iterator__ initially unintuitive, but I'm not sure how you got that from reading the testcase.

> and this comment:
> http://developer.mozilla.org/en/docs/Talk:New_in_JavaScript_1.7#Syntax_for_replacing_native_iterators.3F

I figured out the problem; the code there was setting __iterator__ to a Generator, not to a function which returns a Generator. I expect this will be a fairly common problem for first-time __iterator__ users with JS 1.7. In general I think people are going to run into initial trouble figuring out the code mechanics needed to support custom iteration using the __iterator__ property.

As for the problem reported here, every time I look at it I think the behavior just might be the correct behavior, but I can never manage to convince myself. I suspect that code demonstrates a bug, although I'm not sure what the difference is between that code and some of the code in the testcases I wrote for iterators. In any case, I believe the code in the original MDC question that spawned this, when fixed, will run into the same problem:

http://whereswalden.com/files/mozilla/js-iter-maybe-bug.html

Jeff

--
Rediscover the Web!
http://snurl.com/get_firefox

Reclaim Your Inbox!
http://snurl.com/get_thunderbird

[Nickolay Ponomarev]

On 7/30/06, Jeff Walden <jwald...@mit.edu> wrote:
> Nickolay Ponomarev wrote:
> > Is this behavior expected? I was confused by this and so were the
> > authors of this test:
> > http://lxr.mozilla.org/mozilla/source/js/tests/js1_7/iterable/basic-for-each.js
>
> I was? ;-) I admit to finding __iterator__ initially unintuitive, but I'm not sure how you got that from reading the testcase.
>

Heh, ok, sorry if you are not confused.

The correct test for the current behavior would be to add the "foo"
property on the iterable and assert that index == 1 at the end of the
test.

Either way, you should make an assertion about the value of |index|
after iteration stops.

> > and this comment:
> > http://developer.mozilla.org/en/docs/Talk:New_in_JavaScript_1.7#Syntax_for_replacing_native_iterators.3F
>
> I figured out the problem; the code there was setting __iterator__ to a Generator, not to a function which returns a Generator. I expect this will be a fairly common problem for first-time __iterator__ users with JS 1.7. In general I think people are going to run into initial trouble figuring out the code mechanics needed to support custom iteration using the __iterator__ property.
>

Ah, duh. I didn't read the __defineGetter__ syntax correctly.

Still, returning a Generator from obj.__iterator__ works the same way
as returning a function that returns a Generator (the python way):
> var obj = {a:1, b:2, __iterator__: function() { yield "a"; yield 2 } }

> for(var i in obj) print(i + "->" + obj[i])
a->1

> As for the problem reported here, every time I look at it I think the behavior just might be the correct behavior, but I can never manage to convince myself. I suspect that code demonstrates a bug, although I'm not sure what the difference is between that code and some of the code in the testcases I wrote for iterators. In any case, I believe the code in the original MDC question that spawned this, when fixed, will run into the same problem:
>
> http://whereswalden.com/files/mozilla/js-iter-maybe-bug.html
>
Huh, so it looks like IDL-defined properties always get enumerated in
a for..in loop, no matter what your custom iterator returns. Looks
like a bug to me.

Nickolay

[Jeff Walden]

Nickolay Ponomarev wrote:
> The correct test for the current behavior would be to add the "foo"
> property on the iterable and assert that index == 1 at the end of the
> test.

Point, and I even did so in the sibling basic-Iterator.js test. :-\ I filed bug 346582 to add such assertions to the tests, and I'm now convinced that current behavior is buggy.

[Nickolay Ponomarev]

On 7/30/06, Jeff Walden <jwald...@mit.edu> wrote:

> Nickolay Ponomarev wrote:
> > The correct test for the current behavior would be to add the "foo"
> > property on the iterable and assert that index == 1 at the end of the
> > test.
>
> Point, and I even did so in the sibling basic-Iterator.js test. :-\ I filed bug 346582 to add such assertions to the tests, and I'm now convinced that current behavior is buggy.
>

Blake pointed me to https://bugzilla.mozilla.org/show_bug.cgi?id=346021

Is the issue with
http://whereswalden.com/files/mozilla/js-iter-maybe-bug.html filed
too?

Nickolay

[Jeff Walden]

Nickolay Ponomarev wrote:
> Blake pointed me to https://bugzilla.mozilla.org/show_bug.cgi?id=346021
>
> Is the issue with
> http://whereswalden.com/files/mozilla/js-iter-maybe-bug.html filed
> too?

I suspect they're the same problem, so I'm inclined to take a wait-and-see attitude. :-)

[savethec...@gmail.com]

I notice this issue got fixed in recent nightlies of Firefox 2.0. But
I'm experiencing a different issue which makes me wonder: is it not
possible to override the iterator on an element's prototype?

I experimented with modifying HTMLCollection.prototype.__iterator__
with my custom code, but when I do so, a for..in loop runs
indefinitely. The first few results are the elements I expected, but
each result thereafter is undefined.

This makes me wonder: how can one distinguish between Foo.prototype as
the ancestor of all instances of Foo *and* Foo.prototype as an ordinary
object? In other words, what if I just want to iterate over all the
properties that are defined in Foo.prototype?

I haven't seen this issue addressed anywhere else. Can someone
enlighten me?

Cheers,
Andrew

[Martin Honnen]

savethec...@gmail.com wrote:

> This makes me wonder: how can one distinguish between Foo.prototype as
> the ancestor of all instances of Foo *and* Foo.prototype as an ordinary
> object? In other words, what if I just want to iterate over all the
> properties that are defined in Foo.prototype?

Well you simply need to choose the object you are interested in in the
for..in loop e.g.
for (var propertyName in Foo.prototype) { }
or
for (var propertyName in someFoo) {}
Or where else do you want to distinguish?

--

Martin Honnen
http://JavaScript.FAQTs.com/

[savethec...@gmail.com]

I've answered my own question, but I'll rephrase it:

Say I've got a var foo that's an instance of object Foo. My concern was
that any change I made to Foo.prototype.__iterator__ (which foo
inherits) would affect the Foo.prototype *instance*, which is rarely
what I would want to happen.

The way around this is to check "this == Foo.prototype" -- which feels
dirty, but works:

Foo.prototype.__iterator__ = function(keysOnly) {
for (var propName in this) {
if (propName == "bar" && this != Foo.prototype) { continue; }
yield propName;
}
throw StopIteration;
};

For introspection's sake, I don't want to hide any properties if I'm
enumerating Foo.prototype.

But this raises another issue: how do you use the "vanilla" for..in
loop within an iterator? The code above won't work, because "for (var
propName in this)" invokes the iterator you're already in, and you'll
get an infinite recursion error. In other words, the native Object
iterator is being shadowed by the custom iterator.

This works, though:

Foo.prototype.__iterator__ = function(keysOnly) {
var iter = Object.prototype.__iterator__.apply(this, arguments);
for (var propName in iter) {
if (propName == "bar" && this != Foo.prototype) { continue; }
yield propName;
}
throw StopIteration;
};

So this is the solution: enumerate using Object.prototype.__iterator__
instead. This also feels dirty.

I might be doing this the hard way -- please let me know if I am -- but
otherwise it seems that this could use some smoothing out. By the time
we hit ES4, this might be solved with the "intrinsic" namespace [1].
If not we'd be faced with a problem: even though Object.prototype would
finally be fair game, Object.prototype.__iterator__ would not. (Though
it would be silly to redefine it in the first place.)

Cheers,
Andrew

[1] http://developer.mozilla.org/es4/proposals/intrinsic_namespace.html