tcl/tk 8.6.12 (built by VS 2022) exec command extremely slow

[shenye...@gmail.com]

I have this very strange problem, and is wondering if anyone has this experience.

I use Visual Studio 2022 to build tcl/tk 8.6.12 with some customized command bulit into tcl. But I have been doing this for quite a while without any problem.

For tcl shell 8.6.12 and 8.6.13(with tk86t.dll loaded), use exec to run an external programs will take more than 3 times of executipn time than using tcl 8.6.0 (from star to end of the external program).

Is there anything I should be aware of for the exec command in tcl 8.6.12 ?

Also I found there is an error during the bulding of 8.6.13, which is related to
"git rev-parse HEAD"
It doesn't seem to affect the final outcome of tcl and tk, so I just ignored it. What is this "git rev-parse HEAD" for anyway ? Should I be cautious about this ?

[Francois Vogel]

Le 01/05/2023 à 11:00, shenye...@gmail.com a écrit :
> I use Visual Studio 2022 to build tcl/tk 8.6.12 with some customized command bulit into tcl. But I have been doing this for quite a while without any problem.
>
> For tcl shell 8.6.12 and 8.6.13(with tk86t.dll loaded), use exec to run an external programs will take more than 3 times of executipn time than using tcl 8.6.0 (from star to end of the external program).

Did you by any chance switch from Win10 to Win11? I have seen something
resembling your report when I did (there was no delay on Win10 Tcl/Tk
built with VS 2022). A 5 seconds (timeout?) delay got introduced in my
app. for no obvious reason (Tcl code unchanged). I didn't go to the
bottom of this problem though, I resolved it in quick and dirty mode by
a judiciously placed update in my script.

> Also I found there is an error during the bulding of 8.6.13, which is related to
> "git rev-parse HEAD"
> It doesn't seem to affect the final outcome of tcl and tk, so I just ignored it. What is this "git rev-parse HEAD" for anyway ? Should I be cautious about this ?

I'm building with VS2022 on Win11 and I'm not seeing an error. Could you
show how exactly you are building (show the commands you output), along
with the the exact error?
From a quick look, "git rev-parse HEAD" is used in makefile.vc to put
the commit hash of the bleeding edge (i.e. HEAD) of the current branch
in the manifest.uuid file. Sounds like we need git in the path for this
to work, perhaps that's the issue you are seeing.

Regards,
Francois

[S-Y. Chen]

No not Win 11. I built and run it in Win 10. And it is not only a delay problrm. If the same external program run for 200 seconds in exec of 8.6.0, it will run about 700 seconds in 8.6.12/8.6.13. So I beleive there is something to do with how exec handle the external programs in the OS (or the opposite way).

I use the following building commands (I need multi-thread in tcl. I remembered I got it from some old version building instruction of tcl)

nmake -nologo -f makefile.vc OPTS=threads MSVCDIR=IDE MACHINE=AMD64 -a
nmake -nologo -f makefile.vc OPTS=dll MSVCDIR=IDE MACHINE=AMD64

The error message related to git is like
>>>>>>>>>>> (begin)
git rev-parse HEAD >>C:\...\Tcltk_8_6_13\tcl\win\..\manifest.uuid
'git' is not an internal or external command, excutable program or batch file
NMAKE : fatal error U1077: 'git rev-parse HEAD >>C:\...\Tcltk_8_6_13\tcl\win\..\manifest.uuid:
return code '0x1'
>>>>>>>>>>> (end)

Regards
Chen

[Christian Werner]

Francois Vogel schrieb am Montag, 1. Mai 2023 um 17:18:54 UTC+2:

> ... (there was no delay on Win10 Tcl/Tk

> built with VS 2022). A 5 seconds (timeout?) delay got introduced in my

> app. for no obvious reason (Tcl code unchanged) ...

Francois,

5 seconds rings an alarm bell in my head et voila I found this weird snippet

https://www.androwish.org/home/file?ci=tip&name=jni/tcl/win/tclWinPipe.c&ln=977-998

Laziness prevented me from writing a ticket in the Tcl core repo. So I believe
that those 5 seconds may be caused by some Win32 non-console process
without a GUI and thus without implicitly making a message queue which can
be waited on by the parent process. But that's only wild guesswork on an even
wilder process management design, to put it mildly.

BR,
Christian

[Francois Vogel]

Le 01/05/2023 à 18:11, S-Y. Chen a écrit :
> No not Win 11. I built and run it in Win 10. And it is not only a delay problrm. If the same external program run for 200 seconds in exec of 8.6.0, it will run about 700 seconds in 8.6.12/8.6.13. So I beleive there is something to do with how exec handle the external programs in the OS (or the opposite way).

OK, so it's not the same problem I have seen. Sorry.

> The error message related to git is like
>>>>>>>>>>>> (begin)
> git rev-parse HEAD >>C:\...\Tcltk_8_6_13\tcl\win\..\manifest.uuid
> 'git' is not an internal or external command, excutable program or batch file
> NMAKE : fatal error U1077: 'git rev-parse HEAD >>C:\...\Tcltk_8_6_13\tcl\win\..\manifest.uuid:
> return code '0x1'
>>>>>>>>>>>> (end)

Seems like you don't have git installed.
TIP #599 has introduced a dependency on fossil or git to get the commit
id of the thing being compiled. The error message you get should
indicate both fossil and git are missing from your environment. If it's
the case: how did you check out the source code? If it's not the case:
please file a bug report here: https://core.tcl-lang.org/tcl/tktnew

Regards,
Francois

[Francois Vogel]

Thanks for the pointer (any reason you point into Androwish instead of
Tcl?) and thanks for the idea. I'll have to figure out how this relates
to the 5 sec issue, and even more why there was a change of behavior
when switching from Win10 to Win11.

Regards,
Francois

[Francois Vogel]

Scrolling down a bit I now see what you meant, the link was slightly
off-position I think, it must rather be:

https://www.androwish.org/home/file?ci=tip&name=jni/tcl/win/tclWinPipe.c&ln=1066-1081

Correct?

Deserves a ticket in Tcl repo, isn't it?

Regards,
Francois

[Christian Werner]

Francois Vogel schrieb am Montag, 1. Mai 2023 um 19:18:37 UTC+2:

> Thanks for the pointer (any reason you point into Androwish instead of

> Tcl?) and thanks for the idea...

since it's not a standard feature but another private and long forgotten hack by me.

BR,
Christian

[Christian Werner]

Francois Vogel schrieb am Montag, 1. Mai 2023 um 19:22:16 UTC+2:

> ... Deserves a ticket in Tcl repo, isn't it?

Yes, indeedly, see

https://core.tcl-lang.org/tcl/tktview/147b28d0436d4267a3d95082339f78980144b004

[S-Y. Chen]

No I didn't install fossil or git. I just pull out the code archive from SourceForge.

Is the problem of slow exec related to this ? Should I install fossil/git and recompile it again ?

Regards
Chen

[Francois Vogel]

Le 02/05/2023 à 05:03, S-Y. Chen a écrit :
> No I didn't install fossil or git. I just pull out the code archive from SourceForge.

OK. No bug then.

> Is the problem of slow exec related to this ?

With 99% chances it's not.

Regards,
Francois

[S-Y. Chen]

Any idea what may cause the slow-down of exec ? Maybe I just take out all the "WaitForInputIdle" command and try again ?

Regards
Chen

[Ralf Fassel]

* "shenye...@gmail.com" <shenye...@gmail.com>

| I have this very strange problem, and is wondering if anyone has this experience.
>
| I use Visual Studio 2022 to build tcl/tk 8.6.12 with some customized
| command bulit into tcl. But I have been doing this for quite a while
| without any problem.
>
| For tcl shell 8.6.12 and 8.6.13(with tk86t.dll loaded), use exec to
| run an external programs will take more than 3 times of executipn time
| than using tcl 8.6.0 (from star to end of the external program).

Could you check whether a stock precompiled 8.6.13
(eg. https://sourceforge.net/projects/magicsplat/files/magicsplat-tcl/)
shows the same slow-down as your version?

A difference of 700s vs 200s as you noted elsethread sounds very
severe. I have difficulties to imagine what exec would do to cause this.

R'

[Christian Gollwitzer]

Am 03.05.23 um 10:51 schrieb Ralf Fassel:

> * "shenye...@gmail.com" <shenye...@gmail.com>
> | I have this very strange problem, and is wondering if anyone has this experience.
>>
> | I use Visual Studio 2022 to build tcl/tk 8.6.12 with some customized
> | command bulit into tcl. But I have been doing this for quite a while
> | without any problem.
>>
> | For tcl shell 8.6.12 and 8.6.13(with tk86t.dll loaded), use exec to
> | run an external programs will take more than 3 times of executipn time
> | than using tcl 8.6.0 (from star to end of the external program).
>
> Could you check whether a stock precompiled 8.6.13
> (eg. https://sourceforge.net/projects/magicsplat/files/magicsplat-tcl/)
> shows the same slow-down as your version?
>

Good idea.

> A difference of 700s vs 200s as you noted elsethread sounds very
> severe. I have difficulties to imagine what exec would do to cause this.
>

My guess would be the antivirus scanner. Maybe there is something in Tcl
which prompts the scanner to check the exec in one version but not the
other. Also, caching can be an issue, so one should make sure that the
shortened time does not occur only on the second run over the same files.

Christian

[Ralf Fassel]

* "shenye...@gmail.com" <shenye...@gmail.com>

| I use Visual Studio 2022 to build tcl/tk 8.6.12 with some customized
| command bulit into tcl. But I have been doing this for quite a while
| without any problem.

Same compiler, 8.6.13 here, with no such problem observed.

| For tcl shell 8.6.12 and 8.6.13(with tk86t.dll loaded), use exec to
| run an external programs will take more than 3 times of executipn time
| than using tcl 8.6.0 (from star to end of the external program).

Another idea would be to wrap the external program in a batch file which
logs the start and end times around the call, so you could see whether
it is indeed 'exec' or the external program taking different runtimes in
both circumstances.

@echo off
time /t < nul > timelog.txt
your command goes here
time /t < nul >> timelog.txt
REM EOF

R'

[S-Y. Chen]

I will take some time to test. Please wait for my result.


Chen

[et99]

I would suggest you post your code that does the exec, just in
case. Apologies if you did, I didn't closely read all the replies.

If it is a virus check, does the .exe file with right click properties
show unblock? When it does on my win 10, it still runs it but does the
virus check (or something that takes many seconds).

Can you substitute your executable with something like

exec cmd /c dir >nul:

and see if there's a difference. The dir should go quite fast.

[Ralf Fassel]

* "S-Y. Chen" <shenye...@gmail.com>

Better make those 'time' commands

time < nul > timelog.txt

(without the /t) since non-interactive /t only shows minutes (no
seconds) on my Windows 10.

I don't know any simple command line program like Unix' 'date' on
Windows, so I'd suggested 'time'. The redirection from < nul is
necessary since time is interactive by default.

HTH
R'

[S-Y. Chen]

I found the problem. It was not caused by tcl. I will explain here.

Using the precompiled tcl does help a lot.
The external program I called with exec is an engineering computation program. I call that program by throwing a script file to the program, which will open en existing data file to process.

Using the precompiled tcl produced two different results. One with normal time and one with about 3 times the span. Be aware that these are two data files but supposedly with totally the same contents (at lease from the users' point of view). The same results were re-produced by my own version of tcl/tk 8.6.13.

I was able to pin down that, even these two data files do have the same contents, one of them does take longer time to open and process in the engineering program. From my experience there should be corruption in its data structure. So I am working to figure out the problem, but that's another story.

Precompiled tcl did take longer time to run with tk86t.dll loaded though.

Sorry for the trouble, and thank you for all the help.

Regards
Chen