Something to consider re ever more byzantine builds
S M Ryan
byzantine the builds become every new release. I don't use the distribution
builds because in my opinion they do various things wrong (like not using
SharedFrameworks in Wish.app). I'm used to having do things all over, but do you
have to make it so hard?
Like why do you have two different tommath.h? And one that just includes
tclTomMath.h? Why not just include tclTomMath.h itself? Now I have to figure out
this tclBN_* crap. I'm sure something else will go wrong after that.
I never did understand the purpose of stubs and why I have to deal with them. I
haven't seen that since using CDC 3300 Algol60 segments. That was thirty years
ago. Forward into the past.
--
I'm not even supposed to be here today.
I ASSURE YOU WE'RE OPEN!
Donal K. Fellows
> I never did understand the purpose of stubs and why I have to deal with them.
A stub-enabled extension can be told the location of all (Tcl/Tk)
library calls that it depends on without needing to know about the
details of the library that is supplying those calls. This allows us
to manage the versioning of the ABI better (stub-enabled extensions
don't need to be recompiled when a new version of Tcl comes out) and
also lets us support providing those libraries in non-standard ways
(enabling the tclkit/starkit distribution model).
Donal.
S M Ryan
"Donal K. Fellows" <donal.k...@man.ac.uk> wrote:
> S M Ryan wrote:
> > I never did understand the purpose of stubs and why I have to deal with
> > them.
>
> A stub-enabled extension can be told the location of all (Tcl/Tk)
> library calls that it depends on without needing to know about the
I just want to get it compiled without having to study irrelevant code.
> details of the library that is supplying those calls. This allows us
> to manage the versioning of the ABI better (stub-enabled extensions
> don't need to be recompiled when a new version of Tcl comes out) and
> also lets us support providing those libraries in non-standard ways
> (enabling the tclkit/starkit distribution model).
I saw all this 30 to 20 years ago on 3300s then 6600s with segments, overlays,
capsules, etc.
Uwe Klein
Where is the difference to linking against glibc 2.0
and running against a newer glibc 2.4 ?
( Though I sometimes would like to be able to link
against 2.4(featureset 2.0 only) )
uwe
Neil Madden
> I'm trying to compile 8.5.4 at the moment and I have to marvel at how more
> byzantine the builds become every new release. I don't use the distribution
> builds because in my opinion they do various things wrong (like not using
> SharedFrameworks in Wish.app). I'm used to having do things all over, but do you
> have to make it so hard?
If you believe the core (Mac OS X) build system is wrong, then have you
considered submitting your changes back to the Tcl patch tracker? That
way, if accepted, you wouldn't have so much of a burden adapting to each
new release.
-- Neil
Ron Fox
than OS-X (yes there are other unixes, and even _gasp_ windows and OMG
VMS too).
I would hazard a guess that the more commonality that can be found
between platform build systems...that result in a working system, the
easier it is to maintain the build system.
I would also hazard a guess that the byzantine nature of the build
system (if it is indeed such I can only judge how build systems evolve
from the ones I've hung on autotools), is due to the differences
between various target systems rather than the commonality.
Just my thoughts, not even worth $.02
Ron.
S M Ryan
Neil Madden <n...@cs.nott.ac.uk> wrote:
I have made my suggestions a number of times some years ago, back before there
was a Tcl patch tracker. Now I am content to do to what I want for myself. I
just wish it wasn't ever more difficult to do so.
miguel
??? We are at SF since 2000-09-05 (https://sourceforge.net/projects/tcl), and
"the first version released was Mac OS X Server 1.0 in 1999, and a
desktop-oriented version, Mac OS X version 10.0 followed in March 2001"
according to http://en.wikipedia.org/wiki/Mac_OS_X
S M Ryan
> The build system also has to accommodate architectures other than
> than OS-X (yes there are other unixes, and even _gasp_ windows and OMG
> VMS too).
> I would hazard a guess that the more commonality that can be found
> between platform build systems...that result in a working system, the
> easier it is to maintain the build system.
Then don't strive for commonality. There is no need, for example, to run
configure on MacOSX. Given the CPU and possibly the OS version, every Mac should
have the same configuration. configure/gnumake/etc is a bad idea taken to
extreme lengths; I'm not going to try to stop other people from jumping off that
cliff: I just wish I wasn't forced to jump off too.
Uwe Klein
Hold my hand. It won't hurt until the very last moment ;-)
Subtly broken ./configure and high falutin requirements for
sgml/whatnot processing ( drawing ever widening circles of package
updates ) just to build a projects docs which happen
to be void of information are a complete turnoff to use or
get involved with some projects.
Helpfull information of the type "if you would be running
debian rosy pig" salvation is just an "apt get upgrit" away
make me scream.
uwe
>
GPS
> I'm trying to compile 8.5.4 at the moment and I have to marvel at how more
> byzantine the builds become every new release. I don't use the distribution
> builds because in my opinion they do various things wrong (like not using
> SharedFrameworks in Wish.app). I'm used to having do things all over, but do you
> have to make it so hard?
It's not generally so hard in Linux and Windows. I build with gcc for
both platforms.
The Mac is probably harder because you have IIRC 3 different ways of
building for that platform. unix/configure, macosx/configure, and the
macosx XCode/project files.
> Like why do you have two different tommath.h? And one that just includes
> tclTomMath.h? Why not just include tclTomMath.h itself? Now I have to figure out
> this tclBN_* crap. I'm sure something else will go wrong after that.
The developers did their best to integrate a big integer library, and
that library is called tommath. Thanks to the work of the people
involved we now have support for very large integers.
> I never did understand the purpose of stubs and why I have to deal with them. I
> haven't seen that since using CDC 3300 Algol60 segments. That was thirty years
> ago. Forward into the past.
The stubs are mostly just indirection with the hope that the function
pointer indirection allows for backward compatiblity. It also costs a
lot in terms of performance I would imagine. Rather than directly doing
a call to a function we end up having to load the address of the
function from the tclStubsPtr, after loading the tclStubsPtr, and so on.
In the tclStubInit.c you can search for "Old" and you will find only
about 2 functions marked "Old." Tk is about the same.
#include <tcl.h>
int Foo_Init (Tcl_Interp *interp) {
Tcl_Obj *obj;
if (NULL == Tcl_InitStubs (interp, TCL_VERSION, 0))
return TCL_ERROR;
obj = Tcl_NewObj();
return TCL_OK;
}
$ gcc -DUSE_TCL_STUBS -I/usr/local/include -E foo.c | less
...
if (((void *)0) == Tcl_InitStubs (interp, "8.6", 0))
return 1;
obj = (tclStubsPtr->tcl_NewObj)();
So what is the cost in terms of machine code I wondered. Here it is:
$ gcc -DUSE_TCL_STUBS -I/usr/local/include foo.c -fverbose-asm -S
movl tclStubsPtr, %eax # tclStubsPtr, tclStubsPtr.0
movl 228(%eax), %eax # <variable>.tcl_NewObj, D.6627
call *%eax # D.6627
movl %eax, -4(%ebp) # D.6628, obj
In comparison to this without -DUSE_TCL_STUBS:
call Tcl_NewObj #
movl %eax, -4(%ebp) # D.6624, obj
The cost is even worse with -fPIC (position independent code (generally
required for an .so)) and -DUSE_TCL_STUBS:
movl tclStubsPtr@GOT(%ebx), %eax #, tmp66
movl (%eax), %eax # tclStubsPtr, tclStubsPtr.0
movl 228(%eax), %eax # <variable>.tcl_NewObj, D.6627
call *%eax # D.6627
movl %eax, -8(%ebp) # D.6628, obj
It would be simpler/easier/faster to just do something like this:
/* Say that 8.4 had a Tcl_DoSomething(Tcl_Interp *interp, int val);
* Then 8.5 adds a new:
* Tcl_DoSomething(Tcl_Interp *interp, int val, int anotherVal);
*/
void Tcl_DoSomething(Tcl_Interp *interp, ...) {
va_list args;
va_start(args, interp);
if (tclExtBinaryVersion.major == 8 && tclExtBinaryVersion.minor < 5) {
int val = va_arg(args, int);
Tcl_DoSomethingOld(interp, val);
} else {
int val = va_arg(args, int);
int anotherVal = va_arg(args, int);
Tcl_DoSomethingNew(interp, args, anotherVal);
}
va_end(args);
}
But, WAIT! That wouldn't work, because you have different extensions in
the same program that need different version requirements, thus the
-DUSE_TCL_STUBS method seems like the only way, unless we were to
radically rework the API, and have it do something like this:
void Tcl_DoSomething(Tcl_Revision rev, Tcl_Interp *interp, ...);
So then you'd have to pass a Tcl_Revision to every API, but that would
be about as complex as the existing -DUSE_TCL_STUBS code...
Long live the Byzantine builds!
George
Alan Anderson
<wyrmwif-DB665F...@sn-ip.vsrv-sjc.supernews.net>,
S M Ryan <wyr...@romeo-alfa-whiskey-bravo-whiskey.com> wrote:
> In article
> <6c47fb59-fa68-4a7c...@79g2000hsk.googlegroups.com>,
> "Donal K. Fellows" <donal.k...@man.ac.uk> wrote:
>
> > S M Ryan wrote:
> > > I never did understand the purpose of stubs and why I have to deal with
> > > them.
> >
> > A stub-enabled extension can be told the location of all (Tcl/Tk)
> > library calls that it depends on without needing to know about...
>
> I just want to get it compiled without having to study irrelevant code.
Then you *do* understand the purpose of stubs.
Cameron Laird
.
.
>> I never did understand the purpose of stubs and why I have to deal
>with them. I
>> haven't seen that since using CDC 3300 Algol60 segments. That was
>thirty years
>> ago. Forward into the past.
>
>
>The stubs are mostly just indirection with the hope that the function
>pointer indirection allows for backward compatiblity. It also costs a
>lot in terms of performance I would imagine. Rather than directly doing
>a call to a function we end up having to load the address of the
>function from the tclStubsPtr, after loading the tclStubsPtr, and so on.
[an abundance of more detail]
.
.
Stubs are very, very important--perhaps a foundation of Tcl's
character. There are occasional days when I think that's a
very bad thing, but mostly I regard it as a great advantage
Tcl enjoys, and that essentially no other language has figured
out yet. Briefly, every other comparable language (maybe not
Lua) has painful arthritis with people doing painfully ugly
things to maintain compiled extensions that work with language
releases of interest. Stubs has almost entirely fulfilled its
promise of eliminating for working extension-writers those
worries.
Cameron Laird
>than OS-X (yes there are other unixes, and even _gasp_ windows and OMG
>VMS too).
.
.
? There's something fishy in this area. Is there truly stuff in
the current build system that contributes to OpenVMS (or QNX or
...) builds? I don't see it, but I recognize I have a bad attitude
about autonconf.
I have plenty of appreciation for how hard a chore generation is; I
spent much of the '80s specializing in it, and have continued to
focus on build systems often since. How can the Tcl build system
possibly be *more* complex than in the past, though? It has given
up on tracking Mac OS 9 or OpenVMS, I think it now aspires to only
one compiler for HP-UX, SCO is of course a thing of the past, ...
I think I'm missing something.
Donal K. Fellows
> Where is the difference [from stubs] to linking against glibc 2.0
> and running against a newer glibc 2.4 ?
> ( Though I sometimes would like to be able to link
> against 2.4(featureset 2.0 only) )
In theory, they could be the same. The awkward issues for Tcl
extensions lie in the fact that we don't want to require the locations
of libraries to be baked in — that would otherwise kill starkits — and
not all systems have a linker as good as the Gnu ld.so. Windows is
notorious this way, but they're not the only one.
By doing it ourselves (with the help of some Tcl scripts of course) we
can make everything work everywhere. The only costs for extension
authors are:
1: Use Tcl_InitStubs(...) instead of Tcl_PkgRequire("Tcl"...)
2: Compile with USE_TCL_STUBS defined
3: Link against the Tcl stub library.
And in fact even step 1 doesn't force you to use stubs; when
USE_TCL_STUBS isn't defined, the Tcl_InitStubs call is changed into
the Tcl_PkgRequire you'd need anyway. (Your code does remember that
bit, yes?)
Donal.
Donal K. Fellows
> The Mac is probably harder because you have IIRC 3 different ways of
> building for that platform. unix/configure, macosx/configure, and the
> macosx XCode/project files.
The Mac actually uses the stuff in unix/ for the main bulk of what it
does, since it's really a Unix platform at heart. The macosx/configure
and Xcode stuff are just wrappers around it. (I know this isn't at all
obvious from a quick glance.)
Donal.
S M Ryan
> spent much of the '80s specializing in it, and have continued to
> focus on build systems often since. How can the Tcl build system
> possibly be *more* complex than in the past, though? It has given
Looking at the Tcl.framework/Versions/8.5/Resources I find a tcl8 directory
where it appears some .tcl have been moved out of the Scripts and renamed. I
don't know why, I don't know where to find an explanation, I don't know what
this signifies. I don't recall this last time I build 8.4. Makefiles have become
essentially unreadable and non-modifiable (try figuring where to put
@executable_path/../SharedFrameworks in the distributed Makefile); I discover
what happens by doing random makes and snagging the output.
This is byzantine. It's taking the poorly designed gnu-style build to extremes.
I don't expect to anyone else to understand this, but at please don't force
everyone else to use the same poorly designed software.
S M Ryan
Alan Anderson <aran...@insightbb.com> wrote:
I understood them on Algol 60 segments on the CDC 3300 in about 1978.
S M Ryan
> that library is called tommath. Thanks to the work of the people
> involved we now have support for very large integers.
I saw no explanation for two different tommath.h. The solution was to rename one
of them to tommath-ignored.h so it wasn't accidentally picked up with -I.
S M Ryan
"Donal K. Fellows" <donal.k...@man.ac.uk> wrote:
The problem is that nothing is obvious without a long study, if then.
S M Ryan
> Stubs are very, very important--perhaps a foundation of Tcl's
> character. There are occasional days when I think that's a
What I do know about stubs is that
/tk/macosx/tkMacOSXBitmap.c:90:28: error: macro "TkpDefineNativeBitmaps" passed
1 arguments, but takes just 0
void
TkpDefineNativeBitmaps(void)
{
And the solution to this is somewhere in the Makefile I cannot hope to decipher.
This is why I don't like stubs: I am forced to deal with them even when I don't
want them.
As far as I can tell, putting stubs in a framework is a useful as paging a disk
block buffer.
schlenk
whiskey.com> wrote:
> > I have plenty of appreciation for how hard a chore generation is; I
> > spent much of the '80s specializing in it, and have continued to
> > focus on build systems often since. How can the Tcl build system
> > possibly be *more* complex than in the past, though? It has given
>
> Looking at the Tcl.framework/Versions/8.5/Resources I find a tcl8 directory
> where it appears some .tcl have been moved out of the Scripts and renamed. I
> don't know why, I don't know where to find an explanation, I don't know what
> this signifies. I don't recall this last time I build 8.4.
http://www.tcl.tk/cgi-bin/tct/tip/189.html
Michael
schlenk
whiskey.com> wrote:
> > The developers did their best to integrate a big integer library, and
> > that library is called tommath. Thanks to the work of the people
> > involved we now have support for very large integers.
>
> I saw no explanation for two different tommath.h. The solution was to rename one
> of them to tommath-ignored.h so it wasn't accidentally picked up with -I.
>
tommath.h), the other is just the parts used by the Tcl library. and
the non supported functions commented out. I didn't grep the sources
but it should not really matter which one you get for building.
Michael
Donal K. Fellows
> The problem is that nothing is obvious without a long study, if then.
That's when you ask here, yes?
Donal.
schlenk
whiskey.com> wrote:
> In article <ih6ro5-lm1....@lairds.us>, cla...@lairds.us (Cameron Laird) wrote:
> > Stubs are very, very important--perhaps a foundation of Tcl's
> > character. There are occasional days when I think that's a
>
> want them.
often a while ago when it was introduced.
>
> As far as I can tell, putting stubs in a framework is a useful as paging a disk
> block buffer.
apps and frameworks this is probably true.
But if your doing cross platform distribution you might think
different. Its a choice of lesser evil, fight with stubs at build time
(usually a non issue if you don't wanna rewrite the makefiles/
configure) or solve the same problem later when distributing an app.
Basically the stubs/starkit thing is the Tcl way to do the same thing
OS X Frameworks and App Bundles do in a cross platform way.
Michael
S M Ryan
schlenk <sch...@uni-oldenburg.de> wrote:
The bn library version prevents compilation due to stubs name mangling.
Arndt Roger Schneider
I don't think Ryan refers to technical details as byzantine,
its rather the under-documented, ill-transparency of it.
This misery generally affect two things
in software products:
Installation,
Builds ... (ahem three: documentation).
Solution:
Throughly document any snipple in the
build process...
(A call to anyone who added fragments to the build process,
stating purpose, requirements, platform and such).
A 80% document will be perfect :--)
-roger
Daniel A. Steffen
whiskey.com> wrote:
> There is no need, for example, to run configure on MacOSX. Given the CPU and possibly the OS version, every Mac should
> have the same configuration. configure/gnumake/etc is a bad idea taken to
> extreme lengths
I'm sorry but that's completely bogus, the current buildsystem
supports building for Mac OS X on 5 different OS releases and 4
different architectures (including multiple architecture combined in
one build), two windowing systems (X11 & Aqua), and two library layout
standards (frameworks or naked dylibs), all of which have differing
API sets available, differing versions of the build tools, and need
different compiler and linker flags...
Have a look at the "Darwin" sections in tcl/unix/configure.in, tcl/
unix/tcl.m4 and tk/unix/configure.in to see all the different
variations that are checked for. In addition most of the Darwin
specific bits in unix/tcl.m4 are in common with the tclconfig/tcl.m4,
which most extensions use (which allows e.g. to build universal
binaries transparently).
If you tried to replicate all this with a different buildsystem you'd
surely end up reinventing at least half of autoconf and gnumake, and
would end up with a separate buildsystem for the Mac OS X platform
that the maintainers without access to Mac OS X would not keep up to
date when adding new source files or configuration options (already an
issue with the native Xcode buildsystem used in the .xcodeproj files
to build tcltest resp tktest, and even that calls out to unix/
configure BTW)
Of course if you want to volunteer to write and maintain such a
buildsystem, nobody is stopping you...
Cheers,
Daniel
Daniel A. Steffen
whiskey.com> wrote:
> In article <ih6ro5-lm1....@lairds.us>, cla...@lairds.us (Cameron Laird) wrote:
> > Stubs are very, very important--perhaps a foundation of Tcl's
> > character. There are occasional days when I think that's a
>
> What I do know about stubs is that
>
> /tk/macosx/tkMacOSXBitmap.c:90:28: error: macro "TkpDefineNativeBitmaps" passed
> 1 arguments, but takes just 0
the only way this could be happening is if you built Tk itself with Tk
stub macros enabled (which would make no sense). Tk stub macros are
only intended for use of the Tk API in extensions.
> And the solution to this is somewhere in the Makefile I cannot hope to decipher.
the solution is rather to not blindly modify the Makefile in ways that
make no sense...
> This is why I don't like stubs: I am forced to deal with them even when I don't
> want them.
>
> As far as I can tell, putting stubs in a framework is a useful as paging a disk
> block buffer.
stubs have nothing to do with frameworks, and everything to do with
the extension binary being able to run under several versions of Tk.
Even a framework embeds the full path to dependent frameworks/
libraries (including version) it is linked against, and cannot run
against different versions without requiring relinking/
install_name_tool massaging
Cheers,
Daniel
S M Ryan
"Daniel A. Steffen" <daniel....@gmail.com> wrote:
> the solution is rather to not blindly modify the Makefile in ways that
> make no sense...
The solution is to slowly track down every -D the makefile adds.
S M Ryan
"Daniel A. Steffen" <daniel....@gmail.com> wrote:
> On Aug 31, 4:27 pm, S M Ryan <wyrm...@romeo-alfa-whiskey-bravo-
> whiskey.com> wrote:
> > There is no need, for example, to run configure on MacOSX. Given the CPU
> > and possibly the OS version, every Mac should
> > have the same configuration. configure/gnumake/etc is a bad idea taken to
> > extreme lengths
>
> I'm sorry but that's completely bogus, the current buildsystem
> supports building for Mac OS X on 5 different OS releases and 4
Most of which is already in the include directory in different names.
I think knowing is better than guessing, and I would rather trust Apple and few
additional system specific includes than use configure to guess, and then patch
the results when it guesses wrong.
> which most extensions use (which allows e.g. to build universal
> binaries transparently).
I can't build universal transparently on XCode 2.4 because all the defines are
passed on the command line with -D* instead of in a file (where I could include
both separated by #ifdef __POWERPC__ or __i386__); apparently I can do that my
10.5 system.
> If you tried to replicate all this with a different buildsystem you'd
> surely end up reinventing at least half of autoconf and gnumake, and
I've already done alternate build systems that are much simpler, because I
didn't make the same fundamental design error: rather than trying to add
scripting to make, I use a script for decisions make cannot handle, write out
the makefile from the script, and use a shell for what it was intended, and the
simple make for what it was intended.
> Of course if you want to volunteer to write and maintain such a
> buildsystem, nobody is stopping you...
I already do.
miguel
> In article <87e60440-98e0-4288...@m3g2000hsc.googlegroups.com>,
> "Daniel A. Steffen" <daniel....@gmail.com> wrote:
>
>> the solution is rather to not blindly modify the Makefile in ways that
>> make no sense...
>
> The solution is to slowly track down every -D the makefile adds.
>
For someone who is thoroughly unknowledgeable: could you tell me what the
problem is? IOW: could you state clearly what you want to achieve (maybe a
valuable RFE lurking in there!), instead of describing how tough it is to do it
the way you are trying to do it?
I could even imagine that we get a --with-specialbuild=smryan option to the
configuration scripts (name meant to be illustrative not disparaging).
Cheers
Miguel
Daniel A. Steffen
whiskey.com> wrote:
> In article <87e60440-98e0-4288-8ada-3ee0efd85...@m3g2000hsc.googlegroups.com>,
> "Daniel A. Steffen" <daniel.stef...@gmail.com> wrote:
>
> > the solution is rather to not blindly modify the Makefile in ways that
> > make no sense...
>
> The solution is to slowly track down every -D the makefile adds.
no, the solution is to use config headers, exactly like the .xcodeproj
projects already do, i.e. use macosx/configure not unix/configure, the
only difference between the two being the use of config headers
instead of DEFS in a makefile.
Cheers,
Daniel
Daniel A. Steffen
whiskey.com> wrote:
> In article <f094d31a-d3e0-40f3-83a3-9eb36ebc1...@x35g2000hsb.googlegroups.com>,
> "Daniel A. Steffen" <daniel.stef...@gmail.com> wrote:
>
> > On Aug 31, 4:27 pm, S M Ryan <wyrm...@romeo-alfa-whiskey-bravo-
> > whiskey.com> wrote:
> > > There is no need, for example, to run configure on MacOSX. Given the CPU
> > > and possibly the OS version, every Mac should
> > > have the same configuration. configure/gnumake/etc is a bad idea taken to
> > > extreme lengths
>
> > I'm sorry but that's completely bogus, the current buildsystem
> > supports building for Mac OS X on 5 different OS releases and 4
>
> Most of which is already in the include directory in different names.
not sure what you are referring to, SDKs? not available on all the OS
releases we support building on...
> I think knowing is better than guessing, and I would rather trust Apple and few
> additional system specific includes than use configure to guess, and then patch
> the results when it guesses wrong.
yeah right, and what do you do when there can be different versions of
the linker present on the same OS release depending on which Xcode
version the user has installed? this has been a problem several times
before, you need something that modifies the linker flags in the
buildsystem to deal with that, trusting include files won't help
there...
> I can't build universal transparently on XCode 2.4 because all the defines are
> passed on the command line with -D* instead of in a file
no they are not if you use the macosx/configure with config headers,
how exactly do you think my .xcodeproj files handle exactly this?
> apparently I can do that my 10.5 system.
apparently myself and many others including ActiveState for the
ActiveTcl distro and teapot use the present system to build tcl, tk
and many extensions universally just fine.
> > Of course if you want to volunteer to write and maintain such a
> > buildsystem, nobody is stopping you...
>
> I already do.
then what are you complaining about exactly? contribute and maintain
your system in the public sources and if it is better than what is
already there, people will use it...
Cheers,
Daniel
morgan mair fheal
"Daniel A. Steffen" <daniel....@gmail.com> wrote:
> then what are you complaining about exactly? contribute and maintain
The byzantine builds that I have to decipher.
arf meow arf - raggedy ann and andy for president and vice
limp and spineless lint for brains is better yet and nice
then rueing pair of shrub and dick the republican lice
call me desdenova seven seven seven seven seven seven