Itcl4: Migration issues

[Uwe Schmitz]

TCLers,

migrating to Tcl8.6 and Itcl4 showed up some
(possibly intended) incompatibilities.

First I discovered that the "@name" notation, for
calling methods implemented in C, is not provided
any more. Fortunately I use it very seldom and
solely for calling built-in "@itcl..." functionality.

I think i can replace the use of
@itcl-builtin-configure
@itcl-builtin-cget
by a direct call or chaining to the original Itcl
methods. So, to the best of may actual knowledge,
I may have a work around here.

But I found no solution for "@itcl <obj> <varName>"
construct. It allows access to all variables of
an arbitrary object ignoring protection levels.
I use it heavily to serialize or dump whole object trees.
Basically you can get information (including the value)
about object properties by doing "<obj> info variable <varName>".
Unfortunately this doesn't help if <varName> is an Tcl array.
Then the value is reported as "<undefined>". Here comes
"@itcl ..." into play. With this construct you can
ask "array exists {@itcl ...}", and the contents can be inquried
as usual by "array get {@itcl ...}".
So if there is no workaround (other than using dicts ;-)),
this seems to be a show stopper for me at the moment.

Second I see a first regression concerning "info level":
<code>
puts "Itcl=[package require Itcl]"
proc infolevel-1 {} {
puts "\[info level -1\] = '[info level -1]'"
}
itcl::class Cls {
public method m1 {} {
infolevel-1
}
}
set obj [Cls #auto]
$obj m1
</code>

Output Tcl8.5/Itcl3:
Itcl=3.4
[info level -1] = 'm1'

Output Tcl8.6/Itcl4:
Itcl=4.1.1
[info level -1] = 'cls0 m1'

With Itcl4 I get "classcommand methodname" where
Itcl3 returns only "methodname". Is this intended?
I use this quite often for logging of lets say
"debugging purposes".

Thanks in advance for any help
Uwe

[Don Porter]

On 04/11/2018 05:44 AM, Uwe Schmitz wrote:
> migrating to Tcl8.6 and Itcl4 showed up some
> (possibly intended) incompatibilities.

Let me say the nice things first. Thanks for pursuing this and
thanks for making your findings public. I don't know how many
people you are helping, but I'm pretty sure it is more than zero.

With that out of the way, I'm sorry to say that all your complaints
appear to be the result of your coding practices going outside of
Itcl's supported interfaces, at least as I understand them. I cannot
offer much hope that Itcl 4 will ever accommodate such code, but
perhaps we can discover some workarounds or "shims" that can support
a migration path for you.

> First I discovered that the "@name" notation, for
> calling methods implemented in C, is not provided
> any more.

This is a false claim. Itcl 4 continues to support the
@name notation to enable methods to be implemented as
C routines. In fact it uses that capability to implement
some of its built-in functionality.

> Fortunately I use it very seldom and
> solely for calling built-in "@itcl..." functionality.
>
> I think i can replace the use of
> @itcl-builtin-configure
> @itcl-builtin-cget
> by a direct call or chaining to the original Itcl
> methods.

So your problems are not the result of any failure
of the @name mechanism, but because your code has had
some reason to intrude on the internal implementation
of the built-in "configure" and "cget" methods instead
of just calling them in the conventional way?

I'm guessing a bit here, because it's not clear to me
what your code is doing or why it ceases to work against
Itcl 4.1, but something your code should not be doing is
making calls based on knowledge of the existence of
@itcl-builtin-cget and the like.

> But I found no solution for "@itcl <obj> <varName>"
> construct.

You should mention that the documentation has told you not
to use that directly for a long long time.

https://www.tcl.tk/man/itcl3.1/scope.n.html

A completely different mechanism is in place in Itcl 4, so
you need to restrict to the recommended use of [itcl::scope]
to be able to migrate. It might be possible to set up
some [upvar] links to fake the old way if it's important enough.

> ... With this construct you can

> ask "array exists {@itcl ...}", and the contents can be inquried
> as usual by "array get {@itcl ...}".
> So if there is no workaround (other than using dicts ;-)),
> this seems to be a show stopper for me at the moment.

Ok, well I can appreciate on one level finding a clever way
to do something the package authors never gave you a proper
interface to do, but that backdoor is now closed. Is the issue
the same one in this ticket?

http://core.tcl.tk/itcl/tktview?name=050ac21f73

Seems to me the path forward is to invent some proper access
to that information, add it to Itcl 3 and Itcl 4, then update
your code to use it, which would then be in a position to migrate.

> Second I see a first regression concerning "info level":
> <code>
> puts "Itcl=[package require Itcl]"
> proc infolevel-1 {} {
> puts "\[info level -1\] = '[info level -1]'"
> }
> itcl::class Cls {
> public method m1 {} {
> infolevel-1
> }
> }
> set obj [Cls #auto]
> $obj m1
> </code>
>
> Output Tcl8.5/Itcl3:
> Itcl=3.4
> [info level -1] = 'm1'
>
> Output Tcl8.6/Itcl4:
> Itcl=4.1.1
> [info level -1] = 'cls0 m1'
>
> With Itcl4 I get "classcommand methodname" where
> Itcl3 returns only "methodname". Is this intended?
> I use this quite often for logging of lets say
> "debugging purposes".

I'm not sure how Itcl 3 managed to get Tcl to lie about
this. I have difficulty imagining a situation where the
Itcl 3 answer is better than the Itcl 4 answer.

In Itcl 4, every Itcl instance is also a TclOO instance,
so it stands to reason they will record themselves on
the execution stack in the same way. I don't think this
can change, but again I don't see what situation you
would want it for.

Hope that helps.

--
| Don Porter Applied and Computational Mathematics Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|

[Uwe Schmitz]

Don,

thanks for your comments. Please find my
replies inline...

Am Freitag, 13. April 2018 22:19:53 UTC+2 schrieb Don Porter:
> On 04/11/2018 05:44 AM, Uwe Schmitz wrote:
> > migrating to Tcl8.6 and Itcl4 showed up some
> > (possibly intended) incompatibilities.
>
> Let me say the nice things first. Thanks for pursuing this and
> thanks for making your findings public. I don't know how many
> people you are helping, but I'm pretty sure it is more than zero.
>
> With that out of the way, I'm sorry to say that all your complaints
> appear to be the result of your coding practices going outside of
> Itcl's supported interfaces, at least as I understand them. I cannot
> offer much hope that Itcl 4 will ever accommodate such code, but
> perhaps we can discover some workarounds or "shims" that can support
> a migration path for you.
>

Yes, you're absolutely right and I'm aware of this "dark places".
I always try to use only documented features, but, as every
programmer knows, sometimes there are situations you're
going to leave the path of virtue...
I try to use the switch to Itcl 4 to get rid
of some of these incompatibilities.

> > First I discovered that the "@name" notation, for
> > calling methods implemented in C, is not provided
> > any more.
>
> This is a false claim. Itcl 4 continues to support the
> @name notation to enable methods to be implemented as
> C routines. In fact it uses that capability to implement
> some of its built-in functionality.
>

Sorry for this misstatement. I mistakenly followed
that from my experiences concerning "@itcl..." syntax.

> > Fortunately I use it very seldom and
> > solely for calling built-in "@itcl..." functionality.
> >
> > I think i can replace the use of
> > @itcl-builtin-configure
> > @itcl-builtin-cget
> > by a direct call or chaining to the original Itcl
> > methods.
>
> So your problems are not the result of any failure
> of the @name mechanism, but because your code has had
> some reason to intrude on the internal implementation
> of the built-in "configure" and "cget" methods instead
> of just calling them in the conventional way?
>
> I'm guessing a bit here, because it's not clear to me
> what your code is doing or why it ceases to work against
> Itcl 4.1, but something your code should not be doing is
> making calls based on knowledge of the existence of
> @itcl-builtin-cget and the like.
>

As I've written above, they can be substituted by using the
original methods via chain'ing or direct call. I only
mentioned it for other "migrators".

> > But I found no solution for "@itcl <obj> <varName>"
> > construct.
>
> You should mention that the documentation has told you not
> to use that directly for a long long time.
>
> https://www.tcl.tk/man/itcl3.1/scope.n.html
>
> A completely different mechanism is in place in Itcl 4, so
> you need to restrict to the recommended use of [itcl::scope]
> to be able to migrate. It might be possible to set up
> some [upvar] links to fake the old way if it's important enough.
>
> > ... With this construct you can
> > ask "array exists {@itcl ...}", and the contents can be inquried
> > as usual by "array get {@itcl ...}".
> > So if there is no workaround (other than using dicts ;-)),
> > this seems to be a show stopper for me at the moment.
>
> Ok, well I can appreciate on one level finding a clever way
> to do something the package authors never gave you a proper
> interface to do, but that backdoor is now closed. Is the issue
> the same one in this ticket?
>
> http://core.tcl.tk/itcl/tktview?name=050ac21f73
>

Yes, this hits the point!

> Seems to me the path forward is to invent some proper access
> to that information, add it to Itcl 3 and Itcl 4, then update
> your code to use it, which would then be in a position to migrate.
>

This would be very nice and I'm able to get rid of "@itcl ..."
construct.

I use it mostly for debugging/logging e.g.:

puts "$this [info level -1]: message..."

which looks a little bit nasty now.
But it isn't a big deal to have a proc infoLevel-1 which
does the "right" thing dependent on Itcl version.

> Hope that helps.
>
Yes. Thank you very much!
Uwe

[Uwe Schmitz]

Don,

thinking about this "@itcl ..." thing...
Wouldn't it be nice to have:

<obj> info variable <varName> -scope

to get the same as calling [itcl::scope <varName>]
inside <obj>? Herewith it's possible to get access
to internal variables and get rid of these
"illegal constructs".

Thanks
Uwe

[Ralf Fassel]

* Uwe Schmitz <uwe.s...@bezreg-koeln.nrw.de>

| thinking about this "@itcl ..." thing...
| Wouldn't it be nice to have:
>
| <obj> info variable <varName> -scope
>
| to get the same as calling [itcl::scope <varName>]
| inside <obj>? Herewith it's possible to get access

| to internal variables [...]

Somewhat completely nukes the concept of OO encapsulation.
Worse, if the returned reference is not write-protected, writing to it
may even violate class constraints about valid values etc...
(yes I know, same issue is with [itcl::scope <varName>], but at least
the class can control who gets that).

If you need this kind of inspection, why not provide an get-accessor in
the class?

My EUR 0.01
R'

[Uwe Schmitz]

Ralf,

> Ralf Fassel:
> * Uwe Schmitz <uwe.schmitz@...>

> | thinking about this "@itcl ..." thing...
> | Wouldn't it be nice to have:
> >
> | <obj> info variable <varName> -scope
> >
> | to get the same as calling [itcl::scope <varName>]
> | inside <obj>? Herewith it's possible to get access
> | to internal variables [...]
>
> Somewhat completely nukes the concept of OO encapsulation.
> Worse, if the returned reference is not write-protected, writing to it
> may even violate class constraints about valid values etc...
> (yes I know, same issue is with [itcl::scope <varName>], but at least
> the class can control who gets that).
>

agreed! For my purpose I only need read access to the variable.
IMHO, this doesn't really hurt encapsulation.
Ok, with the scoped accessor you're able to write to the
variable. But as always: Don't do so if you're not really
sure what you are doing.

> If you need this kind of inspection, why not provide an get-accessor in
> the class?
>

Hmmm, it's not "the class". There are many packages,
some of them have next to hundred classes or so.
Of course, it should be possible to re-define them all
dynamically at runtime, and put a get-acessor in.
But that seems a little over the top for me.

Uwe