uplevel and itcl4.0

[David Rysdam]

I had some discussion of this in a previous thread, but I think I have a
better handle on what's going on now.

Back in 2002, we created a try/catch and tryproc/catch control structure
based on uplevel. We wrote about 250k lines of code with these control
structures in it. It has worked well for us and continues to function
just fine in 8.6 with itcl 3.4. However, it breaks under itcl 4.0.

When I first found this, a few months ago, I decided to just upgrade our
homegrown try/catch to the new native ones. I made some preparatory
steps towards this, but decided to revisit the basic issue again before
I spent any large amount of time on work that could be avoided. I'm now
realizing that using the native try/catch doesn't really eliminate the
problem. We now have other control structures also based on uplevel that
won't work unmodified under itcl4.0 and there are no native replacements
for these.

I've attached[1] a sample program that illustrates the problem. It also
illustrates a potential fix, but at the very least this fix also
involves a lot of work in adding "$this" to every try-embedded method
call in every object. I haven't investigated what problems this might
create with, for example, friends or inherited classes.

#!/usr/bin/env tclsh8.6

package require Itcl 4.0

proc do_uplevel { body } {
uplevel $body
}

itcl::class TestClass {
constructor {} {}

public method helper {}
public method mthis {}
public method m {}
}

proc ::testclass { pathName args } {
uplevel TestClass $pathName $args
}

itcl::body TestClass::helper {} {
puts "TestClass::helper"
}

itcl::body TestClass::mthis {} {
puts "TestClass::mthis"
do_uplevel {
$this helper
}
}

itcl::body TestClass::m {} {
puts "TestClass::m"
do_uplevel {
helper
}
}

set tc [::testclass .]

puts "this works"
do_uplevel {
$tc helper
}
puts "--------"

puts "this also works:"
$tc mthis
puts "--------"

puts "but this does not work:"
$tc m

Conceptually, the "$this" seems superfluous. The body of the do_uplevel
is passed as a string. That string worked fine in the calling context
without being wrapped by do_uplevel, but no longer does when executed by
uplevel. Why not?

[1] Not actually attached. That caused an NNTP error for some reason.

[David Rysdam]

Let me try a shorter post with a simpler example. Why does the below
program fail (error message: "ItclObjectCmd cannot get context object
(NULL)") with Itcl 4.0 when it succeeds with Itcl 3.4?

#!/usr/bin/env tclsh8.6

package require Itcl 4.0

proc do_uplevel { body } {
uplevel $body
}

itcl::class TestClass {
constructor {} {}

public method api-call {}
private method internal-helper {}

}

proc ::testclass { pathName args } {
uplevel TestClass $pathName $args
}

itcl::body TestClass::internal-helper {} {
puts "TestClass::internal-helper"
}

itcl::body TestClass::api-call {} {
puts "TestClass::api-call"
do_uplevel {
internal-helper
}
}
set tc [::testclass .]

$tc api-call

[David Rysdam]

I'm having a lot of NNTP trouble. I've got a bad connection and my
client (icedove) is determined to believe I want to send email.
Hopefully this message gets through this time.

Why does the below program fail (error message: "ItclObjectCmd cannot
get context object (NULL)") with Itcl 4.0 when it succeeds with Itcl 3.4?

#!/usr/bin/env tclsh8.6

package require Itcl 4.0

proc do_uplevel { body } {
uplevel $body
}

itcl::class TestClass {
constructor {} {}

public method api-call {}
private method internal-helper {}

}

proc ::testclass { pathName args } {
uplevel TestClass $pathName $args
}

[tomás zerolo]

David Rysdam <dry...@ll.mit.edu> writes:

> I'm having a lot of NNTP trouble. I've got a bad connection and my
> client (icedove) is determined to believe I want to send
> email. Hopefully this message gets through this time.

I'm seeing your messages. I'm not answering because the question is far
beyond my league.

regards
-- tomás

[David Rysdam]

Thanks. I guess I'll just wait until some knowledgeable hacker has a
moment to look into it. And I'll bump after Christmas in case it's a
vacation thing.

[Don Porter]

How does this problem with Itcl 4 compare with this bug report?

http://core.tcl.tk/tcl/tktview/d8589a4ec50d0423caa0ee5e0f9603223f08b4e5

--
| Don Porter Applied and Computational Mathematics Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|

[David Rysdam]

On 12/30/2013 10:03 AM, Don Porter wrote:
> On 12/17/2013 02:50 PM, David Rysdam wrote:
>> On 12/17/2013 12:43 PM, tomás zerolo wrote:
>>> David Rysdam <dry...@ll.mit.edu> writes:
>>>
>>>> I'm having a lot of NNTP trouble. I've got a bad connection and my
>>>> client (icedove) is determined to believe I want to send
>>>> email. Hopefully this message gets through this time.
>>>
>>> I'm seeing your messages. I'm not answering because the question is far
>>> beyond my league.
>>
>> Thanks. I guess I'll just wait until some knowledgeable hacker has a
>> moment to look into it. And I'll bump after Christmas in case it's a
>> vacation thing.
>
> How does this problem with Itcl 4 compare with this bug report?
>
> http://core.tcl.tk/tcl/tktview/d8589a4ec50d0423caa0ee5e0f9603223f08b4e5
>

That's a slightly different description of what I suspect is the same
bug. It uses the example of one object calling another "through" an
uplevel. It then gives a sensible, albeit unexpected and incorrect,
error. ("Sensible" meaning "intended for programmer consumption".)

My example is an object calling *itself* and the error is an internal
complaint about "context".

However, note the comment from "dkf" on 2013-12-01, which mentions wrong
context and invisible stack frames. That makes sense to me and would
also apply to my issue, I think.

I see the severity is "critical" but it's also assigned to nobody. What
does that translate to probable-days-until-fixed? It's it's short, I can
just wait and see if my thing is fixed. If it's long, maybe I should
submit another bug or add my example to this bug.

[David Rysdam]

On 12/30/2013 10:41 AM, David Rysdam wrote:
> On 12/30/2013 10:03 AM, Don Porter wrote:
>> How does this problem with Itcl 4 compare with this bug report?
>>
>> http://core.tcl.tk/tcl/tktview/d8589a4ec50d0423caa0ee5e0f9603223f08b4e5
>

> I see the severity is "critical" but it's also assigned to nobody. What
> does that translate to probable-days-until-fixed? It's it's short, I can
> just wait and see if my thing is fixed. If it's long, maybe I should
> submit another bug or add my example to this bug.

This bug is kind of a blocker for me, so I thought I'd take a couple
hours and look into it, despite knowing only epsilon about Tcl
internals. Turns out it wasn't too hard of a fix--a one-liner. It fixes
both the above bug and mine, so I guess that proves they were the same
thing after all.

Where do I stick it? I'll also attach a diff in case "here" is the
answer or someone wants to discuss it. (I hope I'm doing this right--git
n00b.)

[David Rysdam]

On 02/10/2014 09:36 AM, David Rysdam wrote:
> This bug is kind of a blocker for me, so I thought I'd take a couple
> hours and look into it, despite knowing only epsilon about Tcl
> internals. Turns out it wasn't too hard of a fix--a one-liner. It fixes
> both the above bug and mine, so I guess that proves they were the same
> thing after all.

By "mine" I of course mean my posted sample program. I have some ways to
go before I know if it fixed my original problem. Also, it's possibly
the case that other functions in that same file need similar
's/framePtr/varFramePtr/' fixes. For instance, I get a similar "cannot
get context" error out of a configbody call to a method...

[Harald Oehlmann]

David,

I have sent a private mail to Arnulf Wiedemann, the author of itcl4, if he could look into it.

Normally, patches should be attached to the tracker item.
Unfortunately, the tracker item is in the wrong tracker.
It should be here:
https://core.tcl.tk/itcl

Eventually, your issue is already fixed there in trunk ?

Regards,
Harald

[David Rysdam]

On 02/10/2014 11:37 AM, Harald Oehlmann wrote:
> Normally, patches should be attached to the tracker item.
> Unfortunately, the tracker item is in the wrong tracker.
> It should be here:
> https://core.tcl.tk/itcl
>
> Eventually, your issue is already fixed there in trunk ?

It may be. I see some code checked in there nearby and a reference to
'uplevel' and a bug #244, but I can't figure out where that bug is
listed to read what it is.

Oh wait, found a link to another bug and my number works in that URL as
well:

http://sourceforge.net/p/incrtcl/bugs/244/

Looks like my bug all right.

I'm putting my patch in place in my system to see if I can move my
migration any farther, but if I encounter problems I'll just wait for
8.6.2 to come out. I assume that's when the referenced itcl 4.0.1 will
be available...

[Don Porter]

This is the same issue raised in an Itcl ticket:

https://sourceforge.net/p/incrtcl/bugs/244/

It was fixed in a recent commit

http://core.tcl.tk/itcl/info/0e02e7b014

The fix ought to be part of Itcl 4.0.1 whenever that gets released.

[Harald Oehlmann]

Thanks Don, tcl ticket closed.

[Harald Oehlmann]

Am Dienstag, 11. Februar 2014 08:34:55 UTC+1 schrieb Harald Oehlmann:
> Thanks Don, tcl ticket closed.

Arnulf wrote me an E-Mail:
- patch did not work
- other fix found and committed to trunk

Hope everything is fine,
Harald

[David Rysdam]

On 02/21/2014 07:49 AM, Harald Oehlmann wrote:
> Am Dienstag, 11. Februar 2014 08:34:55 UTC+1 schrieb Harald Oehlmann:
>> Thanks Don, tcl ticket closed.
>
> Arnulf wrote me an E-Mail:
> - patch did not work
> - other fix found and committed to trunk

When I saw how extensive his patch was, I suspected mine wasn't enough.
I was mainly hoping it would be a spur to action/thought. Which I guess
happened.

Does anyone know about how long I'd have to wait for 8.6.2 to be out?

[Harald Oehlmann]

> Does anyone know about how long I'd have to wait for 8.6.2 to be out?

I don't know, Donald is the release manager.
When interpolating from last year: march 2014

IMHO it would be helpful if you could check if the fix is ok for you.
I suppose, you can also use this itcl dll with tcl 8.6.1.

[David Rysdam]

I took a stab at testing this, but I can't figure out how to even run
configure correctly. I just get an error:

configure: error: cannot find install-sh, install.sh or shtool in
tclconfig "."/tclconfig

Eventually I realized that if this is fix to the Itcl subsystem, maybe
I'm supposed to unpack in my tcl8.6.1/pkgs source tree. But doing that
doesn't make a tclconfig subdir magically appear (like the existing
itcl4.0.0 subdir has), so that didn't work either.

I copied the itcl4.0.0/tclconfig into my new itcl dir and that at least
let the configure script run, but the compilation failed. I don't have
much faith that I'm on the right path.

Exactly how do I plug in the updated itcl source? Or maybe the problem
is that it won't actually compile against/with 8.6.1?

[David Rysdam]

On 03/03/2014 12:30 PM, David Rysdam wrote:
> Exactly how do I plug in the updated itcl source? Or maybe the problem
> is that it won't actually compile against/with 8.6.1?

Or maybe the problem is even more fundamental than that. I have to say I
can't figure out a darn thing about Fossil. I clicked on the "tarball"
link from here:

http://core.tcl.tk/itcl/info/0e02e7b014

[Don Porter]

> I took a stab at testing this, but I can't figure out how to even run
> configure correctly. I just get an error:
>
> configure: error: cannot find install-sh, install.sh or shtool in
> tclconfig "."/tclconfig

http://core.tcl.tk/itcl/wiki?name=Tracking+Itcl+Sources

[David Rysdam]

On 03/04/2014 02:00 PM, Don Porter wrote:
>
>> I took a stab at testing this, but I can't figure out how to even run
>> configure correctly. I just get an error:
>>
>> configure: error: cannot find install-sh, install.sh or shtool in
>> tclconfig "."/tclconfig
>
> http://core.tcl.tk/itcl/wiki?name=Tracking+Itcl+Sources

Sorry for the late reply. Multiple fires at once.

These build instructions worked and the resulting itcl fixes my test
program AND my real program.

Thanks everyone!