problem with Compress plugin Thruk 1.74 and 1.76-2 (haven't tested others)

[thomas blanchard]

Hello all,

I've been fighting against an encoding error with Catalyst v5.90042 and Thruk 1.74 and 1.76-2 , which gave me a loading error on the startup page:

Retrieving http://server/thruk/startup.html?/thruk

Please stand by, Thruks FastCGI Daemon is warming up.failed.
please have a look at the apache error log and the thruk error log.

I figured out from the logs that the FastCGI daemon was actually loaded, so I tried loading http://server/thruk/ and Firefox returned the following error:

Content Encoding Error
The page you are trying to view cannot be shown because it uses an invalid or unsupported form of compression.

Unless I disabled the request for compression in firefox (in about:config, set the key network.http.accept-encoding to an empty value), at which point, I could use Thruk properly.
(and it's not a problem with Firefox, as the same problem appears with Chrome)

Requesting with curl asking for compression gives a page non-uncompressable:

[user@server Thruk-1.76]$ curl -H 'Accept-Encoding: gzip,deflate' http://localhost/thruk/ > ~/tmp/thruk_catalyst-compress.html.gz
[user@server Thruk-1.76]$ file ~/tmp/thruk_catalyst-compress.html.gz
/home/user/tmp/thruk_catalyst-compress.html.gz: data
[user@server Thruk-1.76]$ vim ~/tmp/thruk_catalyst-compress.html.gz
"~/tmp/thruk_catalyst-compress.html.gz" [noeol] 17L, 5373C
Error detected while processing function gzip#read:
line   44:
Error: Could not read uncompressed file
Press ENTER or type command to continue

I suspect the problem to come from the perl5/ library folder which has a lot of old/legacy modules, I've tried to update most of them, especially those linked to the Catalyst Compress plugin, without success.

To solve this problem, I have disabled the Compress plugin in lib/Thruk.pm by removing it from the loaded catalyst plugins:
###################################################
# Set flags and add plugins for the application
[...]
#          Compress
    push @catalyst_plugins, qw/
          Authentication

And added the compression on Apache level adapting from support/apache_fcgid.conf

  <Location /thruk>
    SetOutputFilter DEFLATE
    # Don't compress images
    SetEnvIfNoCase Request_URI \.(?:gif|png|jpg|jpeg)$ no-gzip dont-vary
    Header append Vary User-Agent env=!dont-vary
[...]
  </Location>
  <Location /thruk/cgi-bin/remote.cgi>
    SetOutputFilter DEFLATE
    # Don't compress images
    SetEnvIfNoCase Request_URI \.(?:gif|png|jpg|jpeg)$ no-gzip dont-vary
    Header append Vary User-Agent env=!dont-vary
[...]
  </Location>


[user@server Thruk-1.76]$ file ~/tmp/thruk.html.gz
/home/user/tmp/thruk.html.gz: gzip compressed data, from Unix

A bit more context:
[user@server nagios]$ uname -a
Linux server 2.6.18-308.11.1.el5 #1 SMP Fri Jun 15 15:41:53 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux
[user@server nagios]$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.8 (Tikanga)
[user@server nagios]$ perl -v
This is perl, v5.8.8 built for x86_64-linux-thread-multi


[user@server nagios]$ perl -V
Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.18-274.12.1.el5, archname=x86_64-linux-thread-multi
    uname='linux x86-003.build.bos.redhat.com 2.6.18-274.12.1.el5 #1 smp tue nov 8 21:37:35 est 2011 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Doptimize=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Dversion=5.8.8 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Dprivlib=/usr/lib/perl5/5.8.8 -Dsitelib=/usr/lib/perl5/site_perl/5.8.8 -Dvendorlib=/usr/lib/perl5/vendor_perl/5.8.8 -Darchlib=/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi -Dsitearch=/usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi -Dvendorarch=/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi -Darchname=x86_64-linux-thread-multi -Dvendorprefix=/usr -Dsiteprefix=/usr -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dinc_version_list=5.8.7 5.8.6 5.8.5 -Dscriptdir=/usr/bin'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=define use64bitall=define uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -I/usr/include/gdbm'
    ccversion='', gccversion='4.1.2 20080704 (Red Hat 4.1.2-50)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =''
    libpth=/usr/local/lib64 /lib64 /usr/lib64
    libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.5'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic'


Characteristics of this binary (from libperl):
  Compile-time options: MULTIPLICITY PERL_IMPLICIT_CONTEXT
                        PERL_MALLOC_WRAP USE_64_BIT_ALL USE_64_BIT_INT
                        USE_ITHREADS USE_LARGE_FILES USE_PERLIO
                        USE_REENTRANT_API
  Built under linux
  Compiled at Dec 16 2011 08:20:05
  %ENV:
    PERL5LIB="/home/rt/lib:/home/rt/lib/perl5:/home/nagios/lib:/home/nagios/lib/perl5:/home/nagios/perl5:/home/nagios/perl5/lib/perl5/"
  @INC:
    /home/rt/lib
    /home/rt/lib/perl5
    /home/nagios/lib
    /home/nagios/lib/perl5/x86_64-linux-thread-multi
    /home/nagios/lib/perl5
    /home/nagios/perl5/x86_64-linux-thread-multi
    /home/nagios/perl5
    /home/nagios/perl5/lib/perl5//x86_64-linux-thread-multi
    /home/nagios/perl5/lib/perl5/
    /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.8
    /usr/lib/perl5/site_perl
    /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.8
    /usr/lib/perl5/vendor_perl
    /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi
    /usr/lib/perl5/5.8.8
    .

Hoping this will be useful to someone else, I'm also posting that here so that the errors will come up in searches

Thomas
note: also posted: http://bouktin.blogspot.co.uk/2013/09/problem-catalyst-plugin-compress-thruk.html

[Sven Nierlein]

Hi,

This is a known problem with latest Catalyst Releases which replaced the compression plugin with an internal plugin.
The best idea would be to use the recommended thruk libs from https://github.com/sni/thruk_libs if you really want to install Thruk from source.
Those perl modules would already contain a fix for the bug https://rt.cpan.org/Public/Bug/Display.html?id=87998 in Catalyst-Runtime.

Regards,
Sven

On 06.09.2013 18:28, thomas blanchard wrote:
> Hello all,
>
> I've been fighting against an encoding error with Catalyst v5.90042 and Thruk 1.74 and 1.76-2 , which gave me a loading error on the startup page:
>
> Retrieving http://server/thruk/startup.html?/thruk
>

> /Please stand by, Thruks FastCGI Daemon is warming up.*failed.*
> please have a look at the apache error log and the thruk error log./

>
> I figured out from the logs that the FastCGI daemon was actually loaded, so I tried loading http://server/thruk/ and Firefox returned the following error:
>

> /Content Encoding Error

> The page you are trying to view cannot be shown because it uses an invalid or unsupported form of compression.

> /

> --
> You received this message because you are subscribed to the Google Groups "Thruk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to thruk+un...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[thomas blanchard]

Thanks Sven,

Good to have validation! I'm not crazy nor dumb :-)

Anyway, this is an alternative to using the compression plugin, that may inspire an option in the thruk config to disable the Compress plugin to use Apache's instead.

Thomas

2013/9/6 Sven Nierlein <sv...@nierlein.de>

To unsubscribe from this group and stop receiving emails from it, send an email to thruk+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the Google Groups "Thruk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/thruk/GCQp677sK-E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to thruk+unsubscribe@googlegroups.com.

[Sven Nierlein]

On 9/6/13 21:26, thomas blanchard wrote:
> Thanks Sven,
>
> Good to have validation! I'm not crazy nor dumb :-)
>
> Anyway, this is an alternative to using the compression plugin, that may inspire an option in the thruk config to disable the Compress plugin to use Apache's instead.

Some benchmarks would be interesting. Do you have any idea if using apaches deflate module is somehow faster?

Sven

[thomas blanchard]

I'll try to provide some data if possible, though I can't actually compare with Catalyst's Compress module. I would imagine that using a build in engine in apache to compress would be quicker, but the Compress plugin being more or less a wrapper around C libraries must be quite quick too.

Which version of Catalyst does work with the Compress Plugin?

2013/9/6 Sven Nierlein <sv...@nierlein.de>

[Sven Nierlein]

On 10.09.2013 00:42, thomas blanchard wrote:
> I'll try to provide some data if possible, though I can't actually compare with Catalyst's Compress module. I would imagine that using a build in engine in apache to compress would be quicker, but the Compress plugin being more or less a wrapper around C libraries must be quite quick too.
>
> Which version of Catalyst does work with the Compress Plugin?

All of them? :-) Except the latest release need the patch to work.

Sven