packaging gtk2 script: use Gtk2 -init; failed

[RAPPAZ Francois]

Hi there,

I'm with strawberry perl and try to pack my script invtot.pl with

pp -l "C:\strawberry\perl\site\lib\auto\Cairo\Cairo.dll" -l "C:\strawberry\perl\site\lib\auto\Pango\Pango.dll" -l "C:\strawberry\perl\site\lib\auto\Glib\Glib.dll" -l "C:\strawberry\perl\site\lib\auto\Gtk2\Gtk2.dll" -o invtot.exe invtot.pl

when I run the exe I got
GLib-GObject-CRITICAL **: g_boxed_type_register_static: assertion `g_type_from_name (name) == 0' failed at C:/strawberry/perl/lib/DynaLoader.pm line 223.
cannot register alias Gtk2::Pango::Attribute for the unregistered type (null) at C:/strawberry/perl/lib/DynaLoader.pm line 223.
Compilation failed in require at script/invtot.pl line 9.

Line 9 invokes use Gtk2 -init;

Thanks for any help with this

François

[Roderich Schupp]

On Fri, Feb 10, 2012 at 15:11, RAPPAZ Francois <francoi...@unifr.ch> wrote:
> pp -l "C:\strawberry\perl\site\lib\auto\Cairo\Cairo.dll" -l "C:\strawberry\perl\site\lib\auto\Pango\Pango.dll" -l "C:\strawberry\perl\site\lib\auto\Glib\Glib.dll" -l "C:\strawberry\perl\site\lib\auto\Gtk2\Gtk2.dll" -o invtot.exe invtot.pl
>
> when I run the exe I got
> GLib-GObject-CRITICAL **: g_boxed_type_register_static: assertion `g_type_from_name (name) == 0' failed at C:/strawberry/perl/lib/DynaLoader.pm line 223.
> cannot register alias Gtk2::Pango::Attribute for the unregistered type (null) at C:/strawberry/perl/lib/DynaLoader.pm line 223.
> Compilation failed in require at script/invtot.pl line 9.

Known problem: [rt.cpan.org #55390] Cannot package with GTK on Windows
Note: ignore anything beyond the original bug report, because it fails
to address the real problem.

AFAIK this is what happens:

(1) The -l options for pp aren't necessary: the DLLs should already
be present in the packed
executable. Please pack without -l and run "unzip -l invtot.exe" to verify.

(2) The real problem is that these "glue" DLLs reference each other.
This can't work, as PAR::Packer
extracts these DLLs using a mangled name. It overrides Dynaloader, so
that loading these DLLs
*from perl* (e.g. "use Glib;" -> Glib::bootstrap) works, but not on
the Windows level.
I think that's the reason for error "Compilation failed in require at
Gtk2.pm line 31" in #55390.

If you add the above -l options then your packed executable contains a
second copy of these DLLs
in a directory that is extracted without mangling. Now runtime loading
succeeds, but apparently
the Glib etc initialization code isn't run (or is run in the other
copy of the DLL? is that even
possible on Windows?). That results in "GLib-GObject-CRITICAL **:
g_boxed_type_register_static: assertion
`g_type_from_name (name) == 0".

BTW, where did you get the Gtk stuff from?

Cheers, Roderich

[RAPPAZ Francois]

Hi Roderich

Thanks for replying.

Running
pp -c --lib c:\strawberry\perl\vendor\lib -o invtot.exe invtot.pl

produces the same list of 1358 files in the archive.

Now the error message is somewhat different when I run the exe:
Can't load 'C:\Windows\temp\par-rappazf\cache-56e9d7591c583b009e3620ebf2e703c37dcb76eb\406bec98.dll' for module Pango: load_file:The specified module could not be found at C:/strawberry/perl/lib/DynaLoader.pm line 200. at C:/strawberry/perl/site/lib/PAR/Heavy.pm line 120
Compilation failed in require at Gtk2.pm line 31.
BEGIN failed--compilation aborted at Gtk2.pm line 31.

Compilation failed in require at script/invtot.pl line 9.

BEGIN failed--compilation aborted at script/invtot.pl line 9.

I got the Gtk2 with http://code.google.com/p/gtk-perl-compiler/ which takes the glib stuff from http://ftp.gnome.org/pub/gnome/binaries/win32

Now looking at the bug you mentioned, shall I try the solution proposed there ??

(downgrade to Module::ScanDeps 0.73 using:
cpan> install S/SM/SMUELLER/Module-ScanDeps-0.73.tar.gz
and use
pp -r -o gtk_test.exe gtk_test.pl ^ --lib c:\strawberry\perl\vendor\lib ^ -l c:\strawberry\perl\site\lib\auto\Cairo\Cairo.dll ^ -l c:\strawberry\perl\site\lib\auto\Glib\Glib.dll ^ -l c:\strawberry\perl\site\lib\auto\Pango\Pango.dll ^ -l c:\strawberry\perl\site\lib\auto\Gtk2\Gtk2.dll)

Best

François

[RAPPAZ Francois]

No way to get the packaging working then ?

:-(

Best François

> -----Original Message-----
> From: Roderich Schupp [mailto:roderic...@googlemail.com]
> Sent: mardi, 14. février 2012 10:38
> To: RAPPAZ Francois
> Subject: Re: packaging gtk2 script: use Gtk2 -init; failed
>

> On Tue, Feb 14, 2012 at 09:20, RAPPAZ Francois
> <francoi...@unifr.ch> wrote:
> > I got the Gtk2 with http://code.google.com/p/gtk-perl-compiler/ which
> takes the glib stuff from http://ftp.gnome.org/pub/gnome/binaries/win32
>

> I'll have a look tonite.

>
> > Now looking at the bug you mentioned, shall I try the solution
> proposed there ??
> > (downgrade  to Module::ScanDeps 0.73 using:
>

> No, that won't help.
>
> Cheers, Roderich

[Roderich Schupp]

On Tue, Feb 14, 2012 at 10:43, RAPPAZ Francois <francoi...@unifr.ch> wrote:
> No way to get the packaging working then ?

IIRC the last time this came up, the real problem was that some "glue"
DLLs reference others.
This can't be made to work, ever (unless someone rewrites PAR::Packer
to not mangle DLL names
on extraction). But maybe this has changed (it was a Windows only
phenomenon, anyway).
As I said, I'll have a look, but don't hold your breath.

Cheers, Roderich

[Roderich Schupp]

On Tue, Feb 14, 2012 at 11:08, Roderich Schupp
<roderic...@googlemail.com> wrote:
> But maybe this has changed (it was a Windows only
> phenomenon, anyway).
> As I said, I'll have a look, but don't hold your breath.

I downloaded the latest gtk+-bundle from
http://ftp.gnome.org/pub/gnome/binaries/win32
and build the latest Gtk2 and dependencies from CPAN.

The problem is still there: for example .../auto/Pango/Pango.dll
references symbols
from .../auto/Cairo/Cairo.dll and .../auto/Flib/Glib.dll - so
executive summary is:
it will not work.

TL;DR: On Linux (or any ELF-based system)
you can get away with this: have unresolved symbols in the DLL, but
don't mention
the resolving shared library in the NEEDED list. As long as you manage that
the resolving library has been loaded _before_ Pango.dll is loaded, then the
symbol will be resolved. And Pango.pm does "use Cairo" which causes Cairo.dll
to be loaded before its own bootstrap causes Pango.dll to be loaded.

On Windows, you can't have unresolved symbols unless you explicitly specify
the DLL from which the should be resolved. Hence Pango.dll lists Cairo.dll as
a prerequiste DLL. By the same argument as above, Cairo.dll has already been
loaded, so it works. However, in a packed executable Cairo.dll has
been extracted
into the cache area using a mangled name like 406bec98.dll. This has
indeed been loaded
(because PAR::Packer intercepts Dynaloader's request to load Cairo.dll and
extracts it - if necessary - and loads 406bec98.dll instead).
Apparently the Windows runtime
loader "remembers" the already loaded DLLs using the filename as the
key, hence does not know that 406bec98.dll is the same as Cairo.dll
The result is a pop-up "Can't find Cairo.dll".

BTW, [rt.cpan.org #55390] suggests downgrading Module::ScanDeps
and may have a valid point: apparently packing a script that
contains "use Gtk2" does _not_ imply packing Cairo.pm and Cairo.dll.
I'll fix that up in Module::ScanDeps. But even if I pack using "pp -M Cairo .."
(and checking with "unzip -l packed.exe" that the whole dependency
tree of Gtk2.pm has been packed), the above problem persists.

Cheers, Roderich

[Roderich Schupp]

On Thu, Feb 16, 2012 at 05:40, Konovalov, Vadim (Vadim)** CTR **
<vadim.k...@alcatel-lucent.com> wrote:
> This "Cairo.dll" - is it part of Perl CPAN module?

Yes. The module is called - surprise, surprise - Cairo.

> if YES - then proper CPAN perl module should be updated to improve its
> logic (what is it, BTW?)

Cairo is not the one referencing symbols from other "glue" DLLs,
AFAICT Pango and Gtk2 are the "culprits". There's probably a reason
why they do this,
but take it to their respective maintainers. But since it works outside of
a packed executable, my guess is that they have little inclination to change it.
Esp. since it's PAR::Packer that's to blame here - because PAR::Packer literally
modifies DynaLoader in order to load DLLs using mangled names.

> if - NOT - then the problem could be resolved by adding  needed
> DLLs into the PAR executable with "-l" - it does not renames them, AFAIK.

No, this has been tried before and doesn't work. To repeat myself:
you end up with two copies of the same DLL and _both_ get loaded.
Then some functions are executed in one copy and some in the other
(or perhaps its global variables). This screws up the glib/gobject
initialization
and leads to immediate program death.

Cheers, Roderich

[Roderich Schupp]

On Thu, Feb 16, 2012 at 09:18, Konovalov, Vadim (Vadim)** CTR **
<vadim.k...@alcatel-lucent.com> wrote:
> I am not fully agree here.
>
> PAR is well recognized technique, and if you have a fix to CPAN module
> that allows PAR usage to the said module, the chances to get the module
> updated are high.

As the current PAR maintainer I beg to differ: PAR is a gross hack.

> fortunately you've found root of a problem, you need to nail down
> the exact module in a problem.
> Is it Pango CPAN module?

At least Pango and Gtk2.

>> This dual DLL loading - due to some Perl module misinterprets whether
> XXX.dll already loaded - should be fixed.

"Perl" doesn't misinterpret anything here.
Do you understand how DynaLoader and DLL loading on the OS level works?

> IOW - how exactly it computed that "Cairo.dll" is already loaded?

I don't know - it's an educated guess. I'm not too familiar with the internals
how Windows loads DLLs. Feel free to use something like sysinternals
(http://technet.microsoft.com/en-us/sysinternals/bb842062)
to explore what's really going on.

Cheers, Roderich

[Konovalov, Vadim (Vadim)** CTR **]

> From: Roderich Schupp

> On Thu, Feb 16, 2012 at 09:18, Konovalov, Vadim wrote:
> > I am not fully agree here.
> >
> > PAR is well recognized technique, and if you have a fix to
> CPAN module
> > that allows PAR usage to the said module, the chances to
> get the module
> > updated are high.
>
> As the current PAR maintainer I beg to differ: PAR is a gross hack.

I see the way to resolve who is correct here - try it.

Come with a working patch to the CPAN module author and see what
happens next.

I can put my efforts to verbally convince them to apply your patch, given
that your patch improves PAR situation.

> >> This dual DLL loading - due to some Perl module
> misinterprets whether
> > XXX.dll already loaded - should be fixed.
>
> "Perl" doesn't misinterpret anything here.

why do you think so?

Rather, what do you mean by ""Perl" doesn't misinterpret anything here"?

It could be that some module XXX has logic:

if (something_was_already_loaded()) {
# skip this and that
...

}



> Do you understand how DynaLoader and DLL loading on the OS
> level works?

I dealt with DynaLoader several times

I happen update Dynaloader somehow in long past to fix something that
I can not remember right now, yet I proposed some patches to p5p,
which are applied to perl core, but some of them are far from perfect,
unfortunately.

yet, I have my own stripped+modified version of dynaloader in my dark
corner to bootstrap my own Perl+tcl/tk+win32 technique, alternate to PAR.

>
> > IOW - how exactly it computed that "Cairo.dll" is already loaded?
>
> I don't know - it's an educated guess. I'm not too familiar
> with the internals
> how Windows loads DLLs

I have another educated guess: DynaLoader.pm tracks loaded DLLs in
@dl_shared_objects file:

push(@dl_shared_objects, $file); # record files loaded

See if Pango / Gtk2 munges with it.

Vadim.

[Konovalov, Vadim (Vadim)** CTR **]

> On Tue, Feb 14, 2012 at 11:08, Roderich Schupp
> <roderic...@googlemail.com> wrote:
> > But maybe this has changed (it was a Windows only
> > phenomenon, anyway).
> > As I said, I'll have a look, but don't hold your breath.
>
> I downloaded the latest gtk+-bundle from
> http://ftp.gnome.org/pub/gnome/binaries/win32
> and build the latest Gtk2 and dependencies from CPAN.
>
> The problem is still there: for example .../auto/Pango/Pango.dll
> references symbols
> from .../auto/Cairo/Cairo.dll and .../auto/Flib/Glib.dll - so
> executive summary is:
> it will not work.
>
> TL;DR: On Linux (or any ELF-based system)
> you can get away with this: have unresolved symbols in the DLL, but
> don't mention
> the resolving shared library in the NEEDED list. As long as
> you manage that
> the resolving library has been loaded _before_ Pango.dll is
> loaded, then the
> symbol will be resolved. And Pango.pm does "use Cairo" which
> causes Cairo.dll
> to be loaded before its own bootstrap causes Pango.dll to be loaded.

... which sometimes faces a problem when different shared libraries (.so)
have symbols with same name, and this problem is harder to resolve.

> Apparently the Windows runtime
> loader "remembers" the already loaded DLLs using the filename as the
> key, hence does not know that 406bec98.dll is the same as Cairo.dll
> The result is a pop-up "Can't find Cairo.dll".

This "Cairo.dll" - is it part of Perl CPAN module?

if YES - then proper CPAN perl module should be updated to improve its
logic (what is it, BTW?)

if - NOT - then the problem could be resolved by adding needed
DLLs into the PAR executable with "-l" - it does not renames them, AFAIK.

(BTW, I haven't PAR-packed Gtk, so please do not count on my opinion)

Vadim.

[Konovalov, Vadim (Vadim)** CTR **]

> From: Roderich Schupp

> Cairo is not the one referencing symbols from other "glue" DLLs,
> AFAICT Pango and Gtk2 are the "culprits". There's probably a reason
> why they do this,
> but take it to their respective maintainers. But since it
> works outside of
> a packed executable, my guess is that they have little
> inclination to change it.
> Esp. since it's PAR::Packer that's to blame here - because
> PAR::Packer literally
> modifies DynaLoader in order to load DLLs using mangled names.

I am not fully agree here.

PAR is well recognized technique, and if you have a fix to CPAN module
that allows PAR usage to the said module, the chances to get the module
updated are high.

> > if - NOT - then the problem could be resolved by adding  needed
> > DLLs into the PAR executable with "-l" - it does not
> renames them, AFAIK.
>
> No, this has been tried before and doesn't work. To repeat myself:
> you end up with two copies of the same DLL and _both_ get loaded.
> Then some functions are executed in one copy and some in the other
> (or perhaps its global variables). This screws up the glib/gobject
> initialization
> and leads to immediate program death.

fortunately you've found root of a problem, you need to nail down
the exact module in a problem.
Is it Pango CPAN module?

This dual DLL loading - due to some Perl module misinterprets whether
XXX.dll already loaded - should be fixed.

IOW - how exactly it computed that "Cairo.dll" is already loaded?

Regards,
Vadim.

[RAPPAZ Francois]

In this thread
http://mail.gnome.org/archives/gtk-perl-list/2005-April/msg00206.html

I read that "Just to point it out again: par *does* build a .exe from my gtk2-perl-app that
runs perfectly fine on another windows box without any perl installed --
which is par's intention. The thing I am not able to do is include the
gtk+-libraries into this par/exe, so they need to be installed on the
"target" machine. This is not really perfect, but with the nice windows
installer that is available for gtk+, I prefer it against switching to
perl/tk or writing my own installer..."

So this mean I could package the perl stuff with pp, and run the archive on another machine providing that I have installed Gtk before ?

(I didn't try before asking, sorry)

François

> -----Original Message-----
> From: Roderich Schupp [mailto:roderic...@googlemail.com]
> Sent: jeudi, 16. février 2012 09:35
> To: Konovalov, Vadim (Vadim)** CTR **
> Cc: RAPPAZ Francois; p...@perl.org
> Subject: Re: packaging gtk2 script: use Gtk2 -init; failed
>

[Roderich Schupp]

On Thu, Feb 16, 2012 at 14:59, RAPPAZ Francois <francoi...@unifr.ch> wrote:
> In this thread
> http://mail.gnome.org/archives/gtk-perl-list/2005-April/msg00206.html

Maybe it worked in 2005 with the versions of Gtk2 etc back then
(perhaps because they didn't have references from one glue DLL to
another back then).
AFAIK PAR::Packer's way to extract and load glue DLLs with mangled names
hasn't changed in ages.

> which is par's intention. The thing I am not able to do is include the
> gtk+-libraries into this par/exe, so they need to be installed on the
> "target" machine.

That's not a problem anymore, use the --link option of pp.
Of course, it's your job to come up with the complete list of DLLs needed :)
I would simply add any DLL in the bin directory of the gtk+-bundle.

> So this mean I could package the perl stuff with pp, and run the archive on another machine providing that I have installed Gtk before ?

With "perl stuff" do you mean: "the script and modules I have written"?
And "providing that I have installed Gtk before" means "I have done a complete
installation of Gtk libraries + Gtk Perl bindings on the target machine"?
Sure.

Cheers, Roderich

[RAPPAZ Francois]

I did this:
- pp -B --lib c:\strawberry\perl\vendor\lib -l "C:\strawberry\perl\site\lib\auto\Cairo\Cairo.dll" -l "C:\strawberry\perl\site\lib\auto\Pango\Pango.dll" -l "C:\strawberry\perl\site\lib\auto\Glib\Glib.dll" -l "C:\strawberry\perl\site\lib\auto\Gtk2\Gtk2.dll" -o invtot_l.par invtot.pl
- unzip the par files
- removed the lib/auto/Gtk2, lib/auto/Cairo lib/auto/Pango from the par unzipped directory
- copied the glade folder which contains my glade generated files in the script directory
- zipped all the directory tree back into invtot.par
- pp -o invtot.exe invtot.par

Now the .exe file run on my pc if it is located in the same folder as the original script.

On an other pc, first the exe failed complaining it missed a libglib-2.0-0.dll
I setup Gtk+ runtime from http://sourceforge.net/projects/gtk-win/
Now the line "use Gtk2 -init;" seems to pass. The exe failed latter because it miss the glade files.

Placing the glade folder and files under the script folder (see above) does not work... But the problem of the dll seems to be solved

François

> -----Original Message-----
> From: Roderich Schupp [mailto:roderic...@googlemail.com]
> Sent: vendredi, 17. février 2012 09:04
> To: RAPPAZ Francois
> Subject: Re: packaging gtk2 script: use Gtk2 -init; failed
>

> On Thu, Feb 16, 2012 at 16:29, RAPPAZ Francois
> <francoi...@unifr.ch> wrote:
> > No, I didn't implied to install the perl binding on the target
> machine. I thought "using the archived package  + Gtk2 set up separatly
> with the installer".
>
> How is that different from your original approach?
> The whole packaging problem _is in the Gtk etc Perl bindings_, ie.
> Gtk2.pm etc
>
> Cheers, Roderich

[Roderich Schupp]

On Fri, Feb 17, 2012 at 13:33, RAPPAZ Francois <francoi...@unifr.ch> wrote:
> I did this:
> - pp -B --lib c:\strawberry\perl\vendor\lib -l "C:\strawberry\perl\site\lib\auto\Cairo\Cairo.dll" -l "C:\strawberry\perl\site\lib\auto\Pango\Pango.dll" -l "C:\strawberry\perl\site\lib\auto\Glib\Glib.dll" -l "C:\strawberry\perl\site\lib\auto\Gtk2\Gtk2.dll" -o invtot_l.par invtot.pl
> - unzip the par files
> - removed the lib/auto/Gtk2, lib/auto/Cairo lib/auto/Pango from the par unzipped directory
> - copied the glade folder which contains my glade generated files in the script directory
> - zipped all the directory tree back into invtot.par
> - pp -o invtot.exe invtot.par
>
> Now the .exe file run on my pc if it is located in the same folder as the original script.

I was a bit puzzled why this works :)

It's because of a little known feature of DynaLoader.pm:
When asked to load the "glue" DLL for module Foo::Bar DynaLoader will
usually look
for a file named "auto\Foo\Bar\Bar.dll" below any directory in @INC.
PAR::Packer extends this search to look for this path also in the zip
that is part of the packed executable. Now you effectively removed these paths
from the packed executable, so this fails. In a last ditch effort DynaLoader
simply punts and asks the OS to load "Bar.dll" (i.e. using its usual
search strategy,
on Windows that means looking in the directories in PATH).

Every DLL you add to the packed executable with "pp -l ..." is flatly packed in
the zip in directory shlib/ARCH and unconditionally extracted in the cache area.
Also the packed executable runs with the cache directory prepended to PATH.
So when asked to load the glue DLL for Gtk2.pm etc, DyneLoader loads Gtk2.dll
from the cache directory. That way we get rid of the second copy of the DLLs
(extracted and loaded with mangled names) and inter-DLL symbol references
work as expected.

> On an other pc, first the exe failed complaining it missed a libglib-2.0-0.dll
> I setup Gtk+ runtime from http://sourceforge.net/projects/gtk-win/
> Now the line "use Gtk2 -init;" seems to pass.

You may add libglib-2.0-0.dll etc to the packed executable as well and then
you won't need to install it on the target machine. Simply add all DLLs
in your Gtk-for-Win32 bin directory with "pp -l ...".

Cheers, Roderich