ResourceSpace Collection Download Issues

[Daniel Stinebaugh]

So we have a good install of Resourcespace up and running, all checks out ok in the check page, and for the most part it's working beautifully!

Due to the number of files we deal with, we are using riofs to mount an s3 bucket (as www-data with full permissions) as our filestore and so far that has been working nicely as well,

However we just enabled collection downloads, and the zip/7z/tar files all come back as a 2kb file. I'll include it's contents below as I've been reading these posts to figure out this issue on my own, and know you're going to ask for it.

For starters here is the versions from Resourcespace I'm using:

ResourceSpace version Trunk (SVN)

ResourceSpace Build 5462

Server platform Apache/2.4.7 (Ubuntu)

PHP version 5.5.9-1ubuntu4

ExifTool version 9.46

FFmpeg version ffmpeg version 2.2.git-66e30a2

ImageMagick/GraphicsMagick version Version: ImageMagick 6.7.7-10 2014-03-06 Q16 http://www.imagemagick.org

Browser user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36

A link to the pastebin of the config.php file we're using:

http://pastebin.com/MpUEws0T

The contents of the downloaded zip file (html?!):

</select></table></table></table>

<div style="border:1px solid black;font-family:verdana,arial,helvetica;position:absolute;top:100px;left:100px; background-color:white;width:400px;padding:20px;border-bottom-width:4px;border-right-width:4px;font-size:15px;color:black;">

<table cellpadding=5 cellspacing=0><tr><td valign=middle><img src="http://54.86.139.206/pages/admin/gfx/cherrybomb.gif" width="48" height="48"></td><td valign=middle align=left><span style="font-size:22px;">Sorry, an error has occurred.</span></td></tr></table>

<p style="font-size:11px;color:black;margin-top:20px;">Please <a href="#" onClick="history.go(-1)">go back</a> and try something else.</p>

<p style="font-size:11px;color:black;">You can <a href="http://54.86.139.206/pages/check.php">check</a> your installation configuration.</p>

<hr style="margin-top:20px;"><p style="font-size:11px;color:black;">/var/www/pages/collection_download.php line 468: fopen(/var/www/include/../filestore/tmp/Col_ID13-original.zip): failed to open stream: No such file or directory</p>

</div>

The error in apache's logs:

stat: cannot stat '/var/www/include/../filestore/tmp/Col_ID13-original.7z': No such file or directory

stat: cannot stat '/var/www/include/../filestore/tmp/Col_ID13-original.tar': No such file or directory

stat: cannot stat '/var/www/include/../filestore/tmp/Col_ID13-original.zip': No such file or directory

I've also tried changing the /var/www/include owner:group to www-data (was owned by root prior) but I still recieve the same end result. I wish the location wasn't truncated as well so I had a better idea of what it's trying to do and where.

Any idea's or help you can suggest?

_____________________________________

This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail (or the person responsible for delivering this documentto the intended recipient), you are hereby notified that any dissemination, distribution, printing or coping of this e-mail, and anyattachment thereto, is strictly prohibited. If you have received this e-mail in error, please respond to the individual sending the message, and permanently delete the original and any copy of any e-mail and printout thereof.

[David Magnus]

I have thought about using S3 for my file store but so many posts say not to use it for production.

The difference from other RS installations is that your using S3.

All your error checking should be around S3.

So have you set up riofs to write to a log file? One limitation of riofs is that it cannot rename folders or append information to a file.

Have you navigated through terminal to the filestore tmp folder to see if the file exists?

Sorry if my post sounds negative. I would love to S3 as my file system as well. 

[Brian Irwin]

Hello,

I ran into some troubles getting the zip functionality to work as well.  I ended up having to install the full version of 7zip:

sudo apt-get install p7zip-full

This does sound like more of a permissions issue though.   

I noticed two $storagedir variables (commented out) in your config.php file - could be the cause of error?:

# Filestore Location

#$storagedir="/var/www/filestore";

#$storagedir="/var/www/export";

It looks as if RS did not know where to put the files, so it was trying to add them in the wrong directory?

You just need one $storagedir variable.  

You are also missing the $storageurl variable.  I have the following in my config:

$storagedir= '/var/www/filestore';

$storageurl= 'http://myserverdotcom/filestore';

I am currently using s3-simple-fuse to mount my s3 bucket to filestore.  It works, but performance is a bit too slow and the developers are no longer maintaining it. I was experimenting with riofs but could never get the mount point parameters figured out.  More specifically getting it to mount with apache user permissions.  Are you mounting riofs using fstab?

Brian-

[Daniel Stinebaugh]

Currently I'm manually mounting riofs with the following command:

sudo riofs --uid=33 --gid=33 --fmode=0777 --dmode=0777 -o "allow_other" -c ~/.config/riofs/riofs.conf.xml ResourceSpace-1-Filestore /var/www/filestore

Where the uid/gid of 33 is the same as the www-data user/group which has full (777) permissions to all files and folders in the filestore.

p7zip-full is the one I installed, but good idea!

I've just set the filestore location again, and also enabled logging, and am recieving some new locations (That's super handy thanks Brian for the keen eye!)

and now am seeing that it might be a cli error:

2014-06-02 22:46:44 CLI command: '/usr/bin/7z' -j '/var/www/filestore/tmp/Col_ID13-original.zip' @'/var/www/filestore/tmp/zipcmd13-original.txt'

2014-06-02 22:46:44 CLI output: Error:

Incorrect command line

2014-06-02 22:46:44 CLI errors:

[Daniel Stinebaugh]

Good eye! 
I've uncommented the filestore location and now at least see the full locatoin in the errors. I've also enabled logging, the results from the logs show something like this:

2014-06-02 23:04:24 CLI command: '/usr/bin/7z' -cf  '/var/www/filestore/tmp/Col_ID2-original.tar' @'/var/www/filestore/tmp/zipcmd2-original.txt'

2014-06-02 23:04:24 CLI output: Error:

Incorrect command line

2014-06-02 23:04:24 CLI errors:

As for riofs I'm currently mounting them manually with a command like so (but will script them into the rc.local file once we have things ironed out):

sudo riofs --uid=33 --gid=33 --fmode=0777 --dmode=0777 -o "allow_other" -c ~/.config/riofs/riofs.conf.xml ResourceSpace-1-Filestore /var/www/filestore

(where the uid/gid is the same as the www-data user)

[Daniel Stinebaugh]

I had already installed the 7zip-full package so unfortunatly that's not the issue. 

Good eye on the commented out filestore, I've uncommented that but am still getting errors, looks like a CLI error when trying to zip the files. No resulting file exists in the temp folder of the filestore. 

I have yet to try adding the storageurl, but hope that wouldn't have anything to do with the zip creation process!

As for riofs, I'm currently mounting it manually with the uid= and gid=  options set to the same user id as the www-data user (in my case 33)

But I do plan on creating a script in rc.local to get it to mount on boot.

On Monday, June 2, 2014 9:36:11 AM UTC-7, Brian Irwin wrote:

[Brian Irwin]

Hello Daniel,

No problem at all!  I wonder if there is still a pesky folder permission/ownership issue going on with the tmp directory.

I have run into troubles in the past where some folders in my ResourceSpace install were still owned by the root user.

A quick way to check would be to use the command line/ssh and cd into /var/www/

then run:

ls -l

to list the ResourceSpace directories to see which user/group owns them.

I had to use the chown command to change ownership to the www-data user on the tmp directory.  I think riofs may have some constraints with folder permissions though... I have been researching switching from s3-simple-fuse to riofs for my install.  I came accross this thread:

https://github.com/skoobe/riofs/issues/29

Not sure if this is applicable to your situation.

Best Regards,

Brian-

[opendev]

I have had this issue before in the past as well. I believe it a
configuration issue. Check to make sure you have the same zip settings
as config.default.php. Depending on your OS some folks have had to
change this setting:

# Zip command to use to create zip archive (uncomment to enable
download of collections as a zip file)
 $zipcommand="zip -j";

 
 $use_zip_extension=false; make it
 $use_zip_extension=true;
 
Now download your Collection.

[Brian Irwin]

Hello Daniel,

This thread is a bit old, but I am just now trying to mount my S3 bucket with Riofs and am running into some issues.

Mounting with the following command: sudo riofs --uid=33 --gid=33 --fmode=0777 --dmode=0777 -o "allow_other" -c ~/.config/riofs/riofs.conf.xml mybucket /var/www/filestore

My files show up and my ResourceSpace (version 6.1) seems to work.  Every now and then I get a random fatal error:

/var/www/html/include/general.php line 108: 

rename(/file/path, /to/another/path)

Operation not permitted

Also, uploads larger than 5mb often fail.

Have you ever encountered these errors with your installation?

Thanks for your time :-)

Brian-