uncompress GZIP file data.

[Nagesh Dudam - Clarion, India]

Hello Friends,

I want to decompress the GZIP compressed xml file. I have a big XML and its size is too big, and now its compressed using GZIP, and I want to decompress it and get the data.

I want to do this using Flex/AS. Can I get the code i.e. Action Script code for decompressing the GZIP file.??
Please provide me if any one found.

I found some links:

http://codeazur.com.br/lab/fzip/
http://www.gzip.org/algorithm.txt
http://weblogs.macromedia.com/mchotin/archives/2004/11/enabling_gzip_c.cfm

This suits the problem : http://board.flashkit.com/board/archive/index.php/t-715335.html


Waiting for you reply.....

Thank You.

Regards,
Nagesh D.

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

[Abdul Qabiz]

If your application is running in browser, gzipped content would be decompressed by browser. You have to just set the right content-encoding in the response-headers...

URLLoader uses Browser's API for all HTTP stuff...

Are you using any other API to load gzip content?

-abdul

--
-abdul
---------------------------------------
http://abdulqabiz.com/blog/
---------------------------------------

[Nagesh Dudam - Clarion, India]

yes my application will be running in a browser.

Also where to set the right content-encoding in the response-headers...??? Abdul, can you please elaborate this....please....

I am using

<mx:HTTPService id="myId"
                url="myGZippedData.GZ"                               
                method="POST" useProxy="false"
                fault="myFaultHandler(event)"
                result="getGZippedData(event)">
   
</mx:HTTPService>

This will send the request and after returning back..

How can get the uncompressed data....say into the ArrayCollection or any datatype....

please tell me...

Thanks & waiting for your reply....
Nagesh D.

[Abdul Qabiz]

Dude, don't access gzip file directly from HTTPService. Rather use a php (or server-side script) that would return the gzip content, specify the Content-Encoding: gzip in response-headers, means before you start sending the content to client you set the headers, one of the headers should Content-Encoding: gzip

What is your server-side environment?

-abdul

[Nagesh Dudam - Clarion, India]

Thanks Abdul...

I got the idea now.... and we are using ASP as a server side scripting. I will check out for the code to uncompress the GZip file data using ASP... and let you know..

Thanks a lot Abdul.. I will get back to you soon...

- Nagesh D

[Abdul Qabiz]

The idea is to use gzip compression facility of server. So keep your files as .xml, either configure ur server or ur script to write data as gzipped compressed, if server/script finds client can accept gzip encoded content. Browser generally sends Accept-Encoding request header with all encoding understood by it.

Here is some PHP Code, generally it's only two lines of code required for gzip compression but there are more checks:-

<?php


function print_gzipped_page()
{

    global $HTTP_ACCEPT_ENCODING;
    if( headers_sent() )
    {
        $encoding = false;
    }
    elseif( strpos($HTTP_ACCEPT_ENCODING, 'x-gzip') !== false )
    {
        $encoding = 'x-gzip';
    }
    elseif( strpos($HTTP_ACCEPT_ENCODING,'gzip') !== false )
    {
        $encoding = 'gzip';
    }
    else
    {
        $encoding = false;
    }

    if( $encoding )
    {
        $contents = ob_get_contents();
        ob_end_clean();
        header('Content-Encoding: '.$encoding);
        print("\x1f\x8b\x08\x00\x00\x00\x00\x00");
        $size = strlen($contents);
        $contents = gzcompress($contents, 9);
        $contents = substr($contents, 0, $size);
        print($contents);
        exit();
    }
    else
    {
        ob_end_flush();
        exit();
    }
}

function gzfile_get_contents($filename, $use_include_path = 0)
{
    //File does not exist
    if( !@file_exists($filename) )
    {    return false;    }
  
    //Read and imploding the array to produce a one line string
   $data = gzfile($filename, $use_include_path);
   $data = implode($data);
   return $data;
}

// At the beginning of each page call these two functions
ob_start();
ob_implicit_flush(0);

// Write data, in this case uncompress a gz file and write data.
echo gzfile_get_contents ("./foo.xml.gz");

// Call this function to output everything as gzipped content.
print_gzipped_page();


?>

[Nagesh Dudam - Clarion, India]

Hi,

Firstly, I was able to uncompress the Gzip file content using the PHP script. But  I want to uncompress the GZipped file content in the client side only. So for this I guess we need to implement the AS code.

Has anyone worked on this before, if yes then please let me know how to do that and if not then please provide any other solution for this.

Waiting for the repponse.

Thanks & Regards,
Nagesh Dudam.

[Abdul Qabiz]

Dude, you didn't get my point. HTTP Servers support gzip compression and browsers can decompress it before passing it to Flash.

So configure your server setting or someway, make it send content as gzipped and set the HTTP headers, as discussed....

This would be far better than writing gzip decompression algorithm in ActionScript...

-abdul

[Nagesh Dudam - Clarion, India]

Hi,

Now I got your point Abdul. But there are some browser issues with this Accept-Encoding header.

Netscape Navigator 3

This browser uses HTTP/1.0. It doesn't send an Accept-Encoding header, thus doesn't request compressed content from a server.

Opera 3.5 does not yet understand compressed content.

[Abdul Qabiz]

Well, which browser sends HTTP/1.0 dude?

I believe, all A-grade browsers are capable of handling gzip data, like IE 6 or Firefox? Correct me, if I am wrong..

[Abdul Qabiz]

You Still support Netscape Navigator 3 and Opera 3? Why? Dude, are you sure you have users using that?

-abdul

[mkokadwar]

Hi Nagesh,

I have worked on compression-decompression of data in one of my
earlier projects. Let me share my experience/observations. It might
prove useful for others also.

We can compress-decompress the data by following two ways.

1. Server compresses - browser decompresses:
We can set some server configuration parameters because of which
server sends compressed text data to browser and browser decompresses
it before passing it to flash player. All handled by server and
browser; no need to change application/UI code to do this.
Not all servers(or their versions) support this.
Browsers with HTTP/1.1 support this

2. Server side code compresses - Flash code decompresses
Here, we write some code to compress data at server side and when swf
receives that compressed data we decompress it before parsing.

There are different types of algorithms available for compression -
decompression. We have to make a choice based on following
Compression-decompression rate VS. time required to compress-
decompress

Also, lot of free code libraries written in Java, PHP, actionscript
etc available on net to do this task.

Check following url. It has java code to compress an xml and
actionscript code to decompress it.
http://fgpwiki.corewatch.net/index.php/LZ77_Compression_for_XML_files

Hope this helps.

- Mahesh Kokadwar

On Oct 29, 5:48 pm, "Nagesh Dudam - Clarion, India"

> > On 10/26/07, *Nagesh Dudam - Clarion, India*
> > <nagesh.du...@clariontechnologies.co.in

> > <mailto:nagesh.du...@clariontechnologies.co.in>> wrote:
>
> > Thanks Abdul...
>
> > I got the idea now.... and we are using ASP as a server side
> > scripting. I will check out for the code to uncompress the GZip
> > file data using ASP... and let you know..
>
> > Thanks a lot Abdul.. I will get back to you soon...
>
> > - Nagesh D
>
> > Abdul Qabiz wrote:
> >> Dude, don't access gzip file directly from HTTPService. Rather
> >> use a php (or server-side script) that would return the gzip
> >> content, specify the Content-Encoding: gzip in response-headers,
> >> means before you start sending the content to client you set the
> >> headers, one of the headers should Content-Encoding: gzip
>
> >> What is your server-side environment?
>
> >> -abdul
>

> >> On 10/26/07, *Nagesh Dudam - Clarion, India* <
> >> nagesh.du...@clariontechnologies.co.in

> >> <mailto:nagesh.du...@clariontechnologies.co.in>> wrote:
>
> >> yes my application will be running in a browser.
>
> >> Also where to set the right content-encoding in the
> >> response-headers...??? Abdul, can you please elaborate
> >> this....please....
>
> >> I am using
>
> >> <mx:HTTPService id="myId"

> >> url="*myGZippedData.GZ*"

>
> >> method="POST" useProxy="false"
> >> fault="myFaultHandler(event)"
> >> result="getGZippedData(event)">
>
> >> </mx:HTTPService>
>
> >> This will send the request and after returning back..
>
> >> How can get the uncompressed data....say into the
> >> ArrayCollection or any datatype....
>
> >> please tell me...
>
> >> Thanks & waiting for your reply....
> >> Nagesh D.
>
> >> Abdul Qabiz wrote:
> >>> If your application is running in browser, gzipped content
> >>> would be decompressed by browser. You have to just set the
> >>> right content-encoding in the response-headers...
>
> >>> URLLoader uses Browser's API for all HTTP stuff...
>
> >>> Are you using any other API to load gzip content?
>
> >>> -abdul
>

> >>> On 10/26/07, *Nagesh Dudam - Clarion, India* <
> >>> nagesh.du...@clariontechnologies.co.in

> >>> <mailto:nagesh.du...@clariontechnologies.co.in>> wrote:
>
> >>> Hello Friends,
>
> >>> I want to decompress the GZIP compressed xml file. I
> >>> have a big XML and its size is too big, and now its
> >>> compressed using GZIP, and I want to decompress it and
> >>> get the data.
>
> >>> I want to do this using Flex/AS. Can I get the code i.e.
> >>> Action Script code for decompressing the GZIP file.??
> >>> Please provide me if any one found.
>
> >>> I found some links:
>
> >>> http://codeazur.com.br/lab/fzip/
> >>> http://www.gzip.org/algorithm.txt

> >>> http://weblogs.macromedia.com/mchotin/archives/2004/11/enabling_gzip_...
> >>> <http://weblogs.macromedia.com/mchotin/archives/2004/11/enabling_gzip_...>

>
> >>> This suits the problem :
> >>> http://board.flashkit.com/board/archive/index.php/t-715335.html
> >>> <http://board.flashkit.com/board/archive/index.php/t-715335.html>
>
> >>> Waiting for you reply.....
>
> >>> Thank You.
>
> >>> Regards,
> >>> Nagesh D.
>
> >>> --
> >>> This message has been scanned for viruses and

> >>> dangerous content by *MailScanner*
> >>> <http://www.mailscanner.info/>, and is

> >>> believed to be clean.
>
> >>> --
> >>> -abdul
> >>> ---------------------------------------
> >>> http://abdulqabiz.com/blog/
> >>> ---------------------------------------
>
> >>> --
> >>> This message has been scanned for viruses and

> >>> dangerous content by *MailScanner*
> >>> <http://www.mailscanner.info/>, and is

> >>> believed to be clean.
>
> >> --
> >> This message has been scanned for viruses and

> >> dangerous content by *MailScanner*
> >> <http://www.mailscanner.info/>, and is

> >> believed to be clean.
>
> >> --
> >> -abdul
> >> ---------------------------------------
> >> http://abdulqabiz.com/blog/
> >> ---------------------------------------
>
> >> --
> >> This message has been scanned for viruses and

> >> dangerous content by *MailScanner*
> >> <http://www.mailscanner.info/>, and is

> >> believed to be clean.
>
> > --
> > This message has been scanned for viruses and

> > dangerous content by *MailScanner* <http://www.mailscanner.info/>,

> > and is
> > believed to be clean.
>
> > --
> > -abdul
> > ---------------------------------------
> >http://abdulqabiz.com/blog/
> > ---------------------------------------
>
> > --
> > This message has been scanned for viruses and

> > dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is

> > believed to be clean.
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is

> believed to be clean.- Hide quoted text -
>
> - Show quoted text -

[Nagesh Dudam]

Hi Mahesh,

Thanks for your reply.

As you say that...

2. Server side code compresses - Flash code decompresses
Here, we write some code to compress data at server side and when swf
receives that compressed data we decompress it before parsing.

==> Here can I know how "swf receive that compressed data from the server"...???
Also I have created one Flex Project. I am using Flex2 and AS3.

Created one new AS file named MapXML.as and used the code given on http://fgpwiki.corewatch.net/index.php/LZ77_Compression_for_XML_files

and included this MapXML.as into MXML file and I am getting "Classes must not be nested." error....

Can I know why this is happening..???

And can I use "test.GZ" instead of "test.cpr"..??

Please reply...

Thanks
Nagesh.

[Nagesh Dudam]

Hi,

I was trying to accept the compressed data from the server. For this, I had to set the Accept-Encoding header and I did it as below.

var myurl:URLRequest = new URLRequest();
myurl.url = "http://mydomain/unCompressGZip.php";
myurl.method = "POST";
var authHeader:URLRequestHeader = new URLRequestHeader("Accept-Encoding","gzip");
myurl.requestHeaders.push(authHeader);
var myurlload:URLLoader = new URLLoader();
myurlload.load(myurl);

And when I send (executed the MXML) request then I got run time error saying....

"The HTTP request header Accept-Encoding cannot be set via ActionScript."

And when I checked for this error I got
http://livedocs.adobe.com/labs/air/1/aslr/runtimeErrors.html

Its says that "You are adding a disallowed HTTP header to an HTTP request."

Please let me know how to send the "Accept-Encoding","gzip" header with the request from Flex / AS....

Waiting for reply..

Also Mahesh kokadwar, I am waiting for your reply.. since that will help me...

Thanks
Nagesh D.

[Abdul Qabiz]

Dude, you can not set that header, that's set by the browser... Since Flash Player Plugin uses browser API to do to HTTP/HTTPS communication.

-abdul

[Nagesh Dudam]

Thanks Abdul.
But can you just tell me how to get the compressed data from the Server. what all I need to do for that.??
I am using your PHP code that you had given..

// Write data, in this case uncompress a gz file and write data.

echo gzfile_get_contents ("mysample.gz");

// Call this function to output everything as gzipped content.
print_gzipped_page();

Please tell me the solution for this....

below is my mxml code: I am calling this function on creationComplete

private function loadAllXMLData():void
{

var myurl:URLRequest = new URLRequest();
myurl.url = "http://mydomain/unCompressGZip.php";
myurl.method = "POST";

var authHeader:URLRequestHeader = new URLRequestHeader("Accept:","gzip");
// Accept-Encoding: x-compress; x-zip
//add the header to request
myurl.requestHeaders.push(authHeader);                
myurlload.load(myurl);           
           
Alert.show(myurlload.data);

}

unCompressGZip.php contains your PHP code..

Thanks
Nagesh.

[Abdul Qabiz]

Are you using the code I gave, complete code or part of it?


I would check the response headers being set by server by running http://server/getData.php (or whatever) in browser, monitor response header using FireBug (firefox extension) or ServiceCapture or Charles...

If I see in response header, server is returning gzip as encoding, that means content is compressed when it reaches to client...

Please let us know your findings..

-abdul

[Abdul Qabiz]

BTW! In my php script, I forgot to add important line that actually compresses it to gzip, I realized while I was looking at headers.... Please find the updated code later in this email

I just wrote some actionscript code to verify, it works :)

ActionScript code:-

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" creationComplete="appCreationCompleteHandler()">
    <mx:Script>
        <![CDATA[
            import flash.net.URLLoader;
            import flash.net.URLRequest;
           
            var loader:URLLoader;
           
            private function appCreationCompleteHandler ():void
            {
                loader = new URLLoader ();
                loader.addEventListener (Event.COMPLETE, loaderResultHandler);
                var request:URLRequest = new URLRequest (" http://localhost/foo.php");
                loader.load(request);
            }
           
            private function loaderResultHandler (event:Event):void
            {
                trace (event.target.data);
            }
        ]]>
    </mx:Script>
</mx:Application>

PHP Code:-

<?PHP
 
// Include this function on your pages

if (substr_count($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip'))
{
    ob_start("ob_gzhandler");
}
else
{
    ob_start();
}

ob_implicit_flush(0);



//Write file content here...

echo gzfile_get_contents ("./foo.xml.gz");
 
// Call this function to output everything as gzipped content.
print_gzipped_page();

 
?>

HTTP headers from my test:-

Response Headers

Date    Wed, 31 Oct 2007 08:16:41 GMT
Server    Apache/2.2.4 (Win32) PHP/5.2.4
X-Powered-By    PHP/5.2.4
Content-Encoding    gzip
Vary    Accept-Encoding
Content-Length    42
Keep-Alive    timeout=5, max=100
Connection    Keep-Alive
Content-Type    text/html

Request Headers

Host    localhost
User-Agent    Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8
Accept    text/xml,application/xml,application/xhtml+xml,text/html;q= 0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language    en-us,en;q=0.5
Accept-Encoding    gzip,deflate
Accept-Charset    ISO-8859-1,utf-8;q= 0.7,*;q=0.7
Keep-Alive    300
Connection    keep-alive

[Nagesh Dudam]

Hi,

Thanks Abdul. In my Headers
        Content-Encoding    gzip
        Vary    Accept-Encoding
are missing...

I used your AS and PHP code and also checked the Headers... I got

Response Headers

Cache-Control    no-cache
Content-Length    4290
Content-Type    text/html
Last-Modified    Wed, 31 Oct 2007 09:03:34 GMT
Accept-Ranges    bytes
Etag    "c4d9b3ea9c1bc81:43c"
Server    Microsoft-IIS/6.0
X-Powered-By    ASP.NET
Date    Wed, 31 Oct 2007 12:40:07 GMT

I guess some server side settings needs to be done.... Please tell me those if any...

Thanks & Regards,
Nagesh D.

[Abdul Qabiz]

Pls update the php code, i sent in previous email. The code I sent couple of days back was not really compressing content as gzip. In fact this code is also messy (specially print...) function...

[Nagesh Dudam]

Hi All,

I have downloaded the FileCompressionTool.zip from

http://livedocs.adobe.com/labs/air/1/devappsflex/help.html?content=QuickStart_Compressing_Files_Flex_1.html

I tried to execute this application... but facing some problems like...

Near the function....  public function compressToFile(src:Object, output:File):void

I am getting this error..   1046: Type was not found or was not a compile-time constant: File.

Please let me know why its giving such error.. and also I did this under Flex 3... and under Flex 3 we have flash.filesystem package...

Thanks for any help..
Nagesh D.

[mrinal wadhwa]

This will work only in AIR .... are you running in AIR ?

Mrinal
http://weblog.mrinalwadhwa.com

[Abdul Qabiz]

You can modify the code (GzipEncoder library by P. Robertson) to work with your case.. It's included in with FileCompressionTool...

Right now it reads and writes to File but it uses utils classes that work with bytearrays.. So you can just use them..


-abdul

[Nagesh Dudam]

Hi
Yes Abdul, you are right, I was doing the same as you said... I will let you know if I get through using this..

Thanks
Nagesh D.

[Nagesh Dudam]

Hi,

Abdul, can you please test my code. please....

I am unable to read the bytes from the stream.... It says bytesAvailable: 46726
and when I do the below...
stream.readBytes(srcBytes, 0, stream.bytesAvailable);  
and check the trace("length: " + srcBytes.length); it gives zero...

I have attched the mxml file......And executed in the debug mode...

I debugged the application and I got the below result....

[SWF] D:\nagesh\Flex3 Projects\MyGZip\bin\MyGZip.swf - 518,790 bytes after decompression
completeHandler: [Event type="complete" bubbles=false cancelable=false eventPhase=2]
parseHeader
isCompressed: false
readByte: 0
bytesAvailable: 46726
readUTFBytes:
toString: [object URLStream]
length: 0


Response Headers

Cache-Control    no-cache
Content-Length    46730
Content-Type    application/x-gzip
Last-Modified    Fri, 26 Oct 2007 14:51:25 GMT
Accept-Ranges    bytes
Etag    "9c2263aedf17c81:43c"
Server    Microsoft-IIS/6.0
Date    Fri, 02 Nov 2007 10:06:13 GMT

Request Headers

Host    mydomain.com

User-Agent    Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8

Accept    text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

Accept-Language    en-us,en;q=0.5
Accept-Encoding    gzip,deflate

Accept-Charset    ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive    300
Connection    keep-alive
If-Modified-Since    Fri, 26 Oct 2007 14:51:25 GMT
If-None-Match    "9c2263aedf17c81:43c"

Please let me know where I am going wrong...

Thanks
Nagesh D.