Porting *nix binary extensions to Win32
Melissa Schrumpf
example extension I just wrote under OS X and compile it on Windows.
Now, there are various documents out there that I found which partially
describe the process. Some walk through most steps and then say "if you
want to do X, you need to use Y," without showing how. Or they explain
how to do it for a specific set of conditions. Now, I'm not always
quick on the uptake of these things. I need specific examples. But I'm
willing to do the foot work. As a matter of fact, I'd be willing to put
together a comprehensive, complete guide with examples, if only I can
get information and figure it all out.
I checked the Wiki, but:
The instructions are very close to what I want, but I got linker errors
with mingw, and compile errors from MSVC about problems with the storage
class, and, since there are no examples of "additional prototypes," I
got errors on my prototypes of other functions. (I can get the specific
errors later if anyone cares.) Also, the comment "If you use Borland,
you'll need to use the old EXPORT macro, too" doesn't really tell me --
someone unfamiliar with it -- HOW or WHERE to use the EXPORT macro.
Basically references building the distribution itself, which is fine,
but, the only way to get from there to "how do I write my own," is to
start with the core code start purging until you get down to a
manageable project. This is doable, but is tantamount to building a
model of the entirety of Tcl/Tk in your head, then tossing out the bits
you don't need, once you determine precisely what it is you don't need.
That could take a month.
Is there a better place to start?
Also, a few questions...
Is there a "porting" extensions guide? Or multiple guides, for the
various compilers? Which leads me to my second question:
Binary compatibility on Win32 seems to be a horrendous mess. There
appear to be four main camps: MSVC, Borland, mingw, and cygwin. From
what I've read, some are not compatible with others, or "can be," if you
do a lot of extra work, but it's almost impossible to find references to
what that extra work might be.
Is it possible to create a Win32 dll that can be used by ANY Tcl/Tk
installation? I had thought so. If so, where can I go to find out how
to do this for any/each of the various compiler camps? Can a
mingw-compiled Tcl use a Borland or MSVC dll (etc., all permutations)?
Can the ActiveState distribution use a mingw-compiled extension? Does
the mingw-compiled extension first require a mingw-compiled Tcl/Tk stubs
file? (When I tried this particular combination, it complained that it
couldn't find symbols.) What are the rules?
Thanks in advance for any help.
--
MKS
Eckhard Lehmann
Melissa Schrumpf wrote:
> Is there a "porting" extensions guide? Or multiple guides, for the
> various compilers? Which leads me to my second question:
If the extension is compliant to the standards, it should be no problem
to port it. There is a template for standard TEA extensions:
http://wiki.tcl.tk/5464
>From there it is best to take Msys/MinGW on windows (as long as the
autotools don't support MSVC - which is planned for the next release).
> Binary compatibility on Win32 seems to be a horrendous mess. There
> appear to be four main camps: MSVC, Borland, mingw, and cygwin. From
> what I've read, some are not compatible with others, or "can be," if you
> do a lot of extra work, but it's almost impossible to find references to
> what that extra work might be.
The resulting binaries are compatible - at least if developed in C (C++
binaries are a question of "namespace mangeling", AFAIK). The compilers
produce different quality, however. I heard that MSVC produces the best
code but I did not recognize any difference to gcc so far.
>
> Is it possible to create a Win32 dll that can be used by ANY Tcl/Tk
> installation? I had thought so. If so, where can I go to find out how
> to do this for any/each of the various compiler camps? Can a
> mingw-compiled Tcl use a Borland or MSVC dll (etc., all permutations)?
Yes, if you use the stubs mechanism: http://wiki.tcl.tk/2?Q=stubs. You
have to be carefully to understand what it does and how to use it.
I frequently compile extensions with msys/MinGW and copy the result to
an ActiveTcl installation, or copy binary ActiveTcl extensions to a
MinGW compiled Tcl installation. There have been no problems at all...
> Can the ActiveState distribution use a mingw-compiled extension? Does
> the mingw-compiled extension first require a mingw-compiled Tcl/Tk stubs
> file? (When I tried this particular combination, it complained that it
> couldn't find symbols.)
Probably the mingw compiled extensions was not using stubs, or
something went wrong during compilation. Hard to say without a concrete
error message...
Eckhard
Victor Wagner
: appear to be four main camps: MSVC, Borland, mingw, and cygwin. From
: do a lot of extra work, but it's almost impossible to find references to
: what that extra work might be.
It is not so horrible. Of four compilers you've listed there are really
only two "camps" - native win32 and cygwin.
Format of .dll itself is standard across all windows. There can be
problems with system libraries which are used by .dll. cygwin uses some
Unix emulation level, so cygwin standard library is incompatible with
native one, and cannot coexist in the one application. MSVC and Mingw32
are compatible. Borland is able to use MSVC-compiled DLL and make DLLs
usable from MSVC/Mingw32.
There is additional issue, that there is NO binary compatibility
between C++ interfaces. Even different versions of same compiler
(say MSVC 6 and MSVC 7.1 or GCC 2.95 and GCC 3.x) use incompatible C++
ABI.
But it is not a problem when you are about making Tcl extension.
Tcl API is pure C one, and pure C ABI is compatible within one OS.
So, just don't forget declare your extension's interface procedures as
extern "C" if you are writing on C++.
Also, on Windows exists such strange thing as "import library" i.e. if
you are linking with dynamic library, you don't link with it directly as
you would do on Linux or Solaris with .so libraries, but use special
library. As far as I remember Borland uses incompatible import libraries
format. This is also non-issue with dynamically loadable extensions. You
don't need import libraries to load .dll with tcl load command.
You just need tclstub static library to link your extension with. Import
libraries are special case of static libraries so Borland issue applies
here. But Mingw and MSVC use same format of static libraries, only
different naming convention. So, if you take tclstub84.lib from
ActiveState Tcl (compiled with MSVC) and just rename it to
libtclstub84.a, you'll be able to link with it using Mingw.
There are also special compiler directives for functions which should be
visible as external DLL api. Again, MSVC and Mingw use same syntax.
You can just use defines from tcl.h.
Note that if you make stub-enabled extension, only you Package_Init
function have to be exported.
: Can the ActiveState distribution use a mingw-compiled extension? Does
Yes.
: the mingw-compiled extension first require a mingw-compiled Tcl/Tk stubs
No. One provided by ActiveState would work just fine, if you manage to
tell gcc that it should be used. Either rename <something>.lib to
lib<something>.a or specify full path on the compiler command line
instead of -l<something> switch.
--
Shaun Deacon
windows :
cheers,
Shaun
Joe English
>
>Okay, so I figured I'd have a go at seeing if I could take the binary
>example extension I just wrote under OS X and compile it on Windows.
>
>Now, there are various documents out there that I found which partially
>describe the process. Some walk through most steps and then say "if you
>want to do X, you need to use Y," without showing how. Or they explain
>how to do it for a specific set of conditions. Now, I'm not always
>quick on the uptake of these things. I need specific examples. But I'm
>willing to do the foot work. As a matter of fact, I'd be willing to put
>together a comprehensive, complete guide with examples, if only I can
>get information and figure it all out.
Building Tcl extensions on Windows has, historically,
been a nightmare. The good news is, it's a lot easier
nowadays than it used to be. The bad news -- as you've
already discovered -- is that there's a lot of out-of-date
documentation out there, and it's rarely clear what instructions
are still operative.
Here's a brief summary of Best Common Practices for Building
Tcl extensions on Windows, circa 2006, as I understand them:
* Use the TEA build system [*]. This is a set of autoconf
macros that figures out most of the details for you.
* Install MSYS (http://www.mingw.org/). This gives you WIN32
ports of all the Posix utilities needed to run autoconf-generated
'configure' scripts.
* You'll also need a compiler. Either MSVC 6 or MinGW will work [**].
MinGW is available at http://www.mingw.org/. MSVC6 is available
if you purchased a copy before Microsoft discontinued it.
Note that even if you're using MSVC6, you still need MinGW
in order to run 'configure' scripts.
* At build-time, compile with "-DUSE_TCL_STUBS" and
link with the Tcl stub library, the same as on other
platforms. [***]
* Declare your package Init routine with the DLLEXPORT
storage qualifier (DLLEXPORT is defined in tcl.h):
int DLLEXPORT Foo_Init(Tcl_Interp *interp)
{
if (!Tcl_InitStubs(interp, TCL_VERSION, 0))
return TCL_ERROR;
/* ... rest of package initialization code here... */
}
This last bit is the only extra thing you need on Windows;
it tells the compiler to make "Foo_Init" publically visible
from the .DLL.
>I checked the Wiki, but:
>
> http://wiki.tcl.tk/9372
>
>The instructions are very close to what I want, but I got linker errors
>with mingw, and compile errors from MSVC about problems with the storage
>class, and, since there are no examples of "additional prototypes," I
>got errors on my prototypes of other functions. (I can get the specific
>errors later if anyone cares.)
What were the errors? (I've seen 'em all, and can usually diagnose
them pretty quickly by now :-)
> Also, the comment "If you use Borland,
>you'll need to use the old EXPORT macro, too" doesn't really tell me --
>someone unfamiliar with it -- HOW or WHERE to use the EXPORT macro.
My advice: don't bother with EXPORT. It's only needed for
older versions of the Borland compiler, and if you're using
an older version of the Borland compiler you can expect a
number of other headaches too (see footnote 2).
Also: a lot of code out there still uses the TCL_STORAGE_CLASS
macro, along with a bunch of C preprocessor jiggery-pokery.
Don't use that either.
> http://wiki.tcl.tk/3424
>
>Basically references building the distribution itself, which is fine,
>but, the only way to get from there to "how do I write my own," is to
>start with the core code start purging until you get down to a
>manageable project. This is doable, but is tantamount to building a
>model of the entirety of Tcl/Tk in your head, then tossing out the bits
>you don't need, once you determine precisely what it is you don't need.
>That could take a month.
>
>Is there a better place to start?
Yes, the Tcl sampleextension, available (only) from CVS,
<URL: http://cvs.sourceforge.net/viewcvs.py/tcl/sampleextension/ >
>Binary compatibility on Win32 seems to be a horrendous mess. There
>appear to be four main camps: MSVC, Borland, mingw, and cygwin. From
>what I've read, some are not compatible with others, or "can be," if you
>do a lot of extra work, but it's almost impossible to find references to
>what that extra work might be.
It's actually not *too* bad nowadays. Strictly speaking,
binary compatibility has never been an issue (at least for C,
C++ is another barrel of worms entirely), it's the linkers,
loaders, and runtime libraries that caused all the problems.
Stick with MSV6 or MinGW and you should be OK.
>Is it possible to create a Win32 dll that can be used by ANY Tcl/Tk
>installation? I had thought so. If so, where can I go to find out how
>to do this for any/each of the various compiler camps? Can a
>mingw-compiled Tcl use a Borland or MSVC dll (etc., all permutations)?
Not entirely sure, but MinGW and MSVC6 are known to be
intercompatible -- a MinGW-compiled extension will work
with an MSVC6-compiled Tcl (e.g., ActiveTcl.), and vice versa.
>Does the mingw-compiled extension first require a mingw-compiled Tcl/Tk
>stubs file?
It shouldn't, at least not anymore. There were some issues
with older GCC ports, but recentish MinGW distributions have
finally gotten this sorted out.
> (When I tried this particular combination, it complained
>that it couldn't find symbols.) What are the rules?
What were the missing symbols? Again, this ought to work.
Footnotes:
[*] The officially recommended way to use TEA is to copy and paste
the contents of the Tcl sampleextension, edit the parts that it says
to edit, and Dont' Worry Your Pretty Little Head about what's
happening under the covers. Nowadays, this approach pretty much
works most of the time.
OTOH, if something breaks, or if you need to do something that's
not covered by TEA, you'll need to figure out what's going on.
This can be painful.
[**] Other compiler options are Cygwin (another WIN32 port
of gcc, which tries to make Windows look like Unix), later
versions of MSVC, and Borland. These can probably be
made to work with varying degrees of effort, but expect
headaches if you use anything other than MSVC6 or MinGW.
[***] The name of the Tcl stub library actually varies from
platform to platform; on Windows it's TCLSTUB8[345].LIB.
The TEA autoconf system figures this out for you.
--Joe English
Victor Wagner
> * You'll also need a compiler. Either MSVC 6 or MinGW will
> work [**]. MinGW is available at http://www.mingw.org/.
> MSVC6 is available if you purchased a copy before Microsoft
> discontinued it. Note that even if you're using MSVC6, you
Why 6? Why not MSVC7.1? I've successfully compiled some extension with
it and use them with Active State Tcl (which was compiled with MSVC6, I
suppose). Of course, shared runtime DLL is named differently, but why
not compile extensions with static runtime?
Most common problem with different runtimes are calling free on pointer,
malloced from DLL in application and vice versa, and passing FILE
structures from DLL to application and vice versa.
Both problems do not affect properly written Tcl extensions, because
they use Tcl_Alloc/Tcl_Free, imported from application via stub table,
and use Tcl channels (importing channel functions same way) for I/O.
Why not freely downloadable Visual C toolkit 2003?
I haven't tried to use it to compile Tcl extensions, mostly because
there is no nmake, and if there is TEA build system I prefer MinGW32
anyway.
--
If I were to walk on water, the press would say I'm only doing it
because I can't swim.
-- Bob Stanfield
paulo....@gmail.com
nmake as well. It's part of them.
Victor Wagner wrote:
>
> ...