2GB Download limit

[Gin]

Hi, I can't seem to download any resource larger than 2GB. That is, it downloads, but only 2GB arrives. I've looked into the config & php.ini, but I can't find anything relevant. Would appreciate any advice.

Regards,
Gin

[Gin]

No one has any ideas?

Gin

[Axel Dörfler]

On Tue, Dec 11, 2012 at 6:59 AM, Gin <spirall...@gmail.com> wrote:
> No one has any ideas?

Which browser are you using?
http://support.microsoft.com/kb/298618

Bye,
Axel.

[Benjamin Bailes]

Same for me after upgrading to the latest SVN.

Thanks,

Ben

[David Dwiggins]

I can't replicate this on my system.

Could someone who is having the problem possibly send me the output of
a phpinfo() command on their system? I have a vague theory, but would
need to know more about the affected systems to say for sure.

-David

> --
>
>

[Gin]

To answer an earlier query, I've tested with FF 14.0.1, Opera 12.11 & IE 8. Same problem with all of them, so I doubt it has anything to do with the browser. Also, during download, note the max size as in the image below. The original file is actually 3 GB.

I don't think I'm using the latest SVN. At least, my ResourceSpace Build is 3717. I've attached my phpinfo().

Thanks for the reply,
Gin

[David Dwiggins]

Still a work in progress, but I have been talking to Ben about this a bit offline, and I think the short answer may be  that, at present, you will not be able to download files larger than 2 GB if you are running PHP on a 32 bit server platform. 

There may be a partial workaround for this. But no matter what I don't think it will be possible to work with files larger than 4GB without a 64 bit server. So my general feeling is that if you expect to work with files this large, you should probably upgrade to 64 bit.

It might be possible to work out a solution that would let RS support up to 4 GB on 32 bit systems, although I think this would require a change to the download mechanism, perhaps in a way that would somewhat reduce the security checks. This would require a bit more study.

I do think that no matter what we may need to tweak the system to make it a bit more clear what these limitations are. This could perhaps happen during upload, and should definitely be a part of the installation check.

[Gin]

Thanks for the reply! This is good to know. Move to 64-bit, it is.

May I suggest this info be written in the system requirements for ResourceSpace?

Regards,
Gin

[Benjamin Bailes]

I tested my production machine from before the SVN update and lo and behold, it's got the same problem.  I could have sworn that I tested this prior to going into production, but have no evidence to back it up.

I would echo Gin's suggestion that a recommendation for a 64 bit OS be made loud and clear if it is indeed the source of this problem.  Anyone wanting to download a collection as a zip would run into this problem at some point.  I'm surprised I have not done so before now.

I will begin building a replacement server with a 64 bit OS.

Thanks,

Ben

[Bill]

Don't think that this is related might be. I have been testing the the latest release and I have an error from my download.php 

/pages/download.php line 10: ob_end_clean() [ref.outcontrol]: failed to delete buffer zlib output compression

My server has zlib.output_compression "ON"

I am sure I can't use gzip/zlib compression as well as flush.

So I would like to know which I should pick. I'd recommend gzip rather than flush, again I have no reasoning other than I have zlib.output_compression on with no problems on an older release. I would like to know the direction and if I would need to change server settings? The other question would be does this change or have something to do with larger file downloads? I currently do not have any files larger that 1GB so I have not been able to test the greater than 2GB issues.

[David Dwiggins]

Hi, Bill,

I think this is related to a fix I made to try to eliminate the
possibility of output buffering as a cause of file download limits. I
hadn't realized that zlib output compression uses the same buffering
infrastructure as the standard PHP output buffers.

I just reverted this change in r4105. Can you give this a try and see
if it resolves your issue?

Thanks,

David

> --
>
>

[David Dwiggins]

I just added a line to the installation check that will test for 32
bit systems and will warn 32 bit users that files larger than 2GB will
not be supported. This is in r4106.

There should perhaps be some code elsewhere in the system that warns
end users on the fly if they try to make a zip file or upload a
resource larger than 2GB. But that's a more complicated fix, and not
one I have time to tackle at the moment.

-David

> --
>
>

[Bill]

Yes this does work again. Thanks.

[Benjamin Bailes]

Though by the time you reach the installation check it is a little late to select a different OS :-(

FYI, 64 bit OS seems to have fixed the problem for me.  All I need to do to get an archived collection download that is larger than 2GB is to select 30 - 40 images.  From that perspective a 64bit OS could be seen as a requirement to run RS effectively.

Thanks, David, for providing an explanation of the problem so quickly!

Ben