Keep Folder Structure / use staticSync.php

[Thorsten K]

Hello ResourceSpace-Users,
I have installed ResourceSpace on PHP 5.5 on Ubuntu server 14.04 via svn and configured it for fit my needs. But now I`m struggling with the following problem:
I want to access my files via normal folder structure and want to track them in ResourceSpace.

In the wiki page I found 2 approaches which would be good:
1. Mapping folders in filestore
2. Use staticSync.php

1. Option is mentioned in wiki.resourcespace.org - StaticSynchronisation:
"It[StaticSync] is not intended as a way to store resources externally in order to increase your storage capability. This should be done by mapping drive(s) to the filestore folder at an operating system level. "
The problem is that it seems that I have no option keep the file name / folder structure in the old style. Instead cryptic names are generated for each resource.

2. Option is described in the wiki as  "This is also a good assistant for legacy systems or incredibly large systems where a strong folder structure exists and there is a necessity for external access to the files."
Sounds perfectly, but I have some problems with it:
1. preview isn`t working anymore. For example MP3-Playback. Also the file-size is 0 and the file isn`t accessable via GUI.
2. Metadata is not created like it would be on web-upload (ex. EXIF-Tool Fields which are mapped to metadata fields)
3. Everytime staticSync.php is executed it will add all resources as new. No files are updated.

I have set
$file_checksums=true;
 


Does anybody use staticsync.php in a similiar way and can me give that customized version?

[Allison M Stec]

1. Folder mapping is a way to populate metadata based on original folder structure. This must be done before running staticsync.

2. Be sure that you are using staticsync without ingest. This will allow you to keep original files in a comfortable folder structure. Any previews or alternative files that are created will be placed in the filestore. This means that ResourceSpace needs to have access to the files in their original location or they will be considered "offline".

Please note that staticsync with ingest=false will assume that folders and files do not move or change names. If either of these things happen ResourceSpace will assume that the old file has been deleted and consider the renamed/moved file to be new.

Why don't you paste your staticsync configuration here so we can have a look!

--
ResourceSpace: Open Source Digital Asset Management
http://www.resourcespace.org
---
You received this message because you are subscribed to the Google Groups "ResourceSpace" group.
To unsubscribe from this group and stop receiving emails from it, send an email to resourcespac...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
--
ResourceSpace Developer
Reseller of Colorhythm's Prismpoint Portal DAM

[Thorsten K]

Am Dienstag, 5. Januar 2016 14:39:50 UTC+1 schrieb Allison Stec:

Why don't you paste your staticsync configuration here so we can have a look!

Of course. I extracted the interesting options below. While I did that I noticed that the $staticsync_ingest parameter you mentioned isn`t set. After do that the output (shown below) looked good. Ressources are added and the preview function is working. But the problem is that the files are now copied to filestore and deleted from their original folder. How can I prevent this?

I looked inside the resource-table and I am suspect of some values that are inserted by staticsync.php:
"ref",    "file_modified",            "file_path","file_checksum","file_size","field8"
"58",    "2016-01-05 22:31:13", ""           ,""                     , NULL     ,"Titel"

Why are no checksums created?
And why is file_path empty?

config.php-Options:
$storagedir = '/media/filestore';
$storageurl = '/resourcespace/filestore';

// Create file checksums? (experimental)
$file_checksums=true;
$metadata_report=true;
$file_checksums_fullfile = true;
$staticsync_ingest = 1;

$staticsync_extension_mapping_default=8; # 8 = Sonstige
$staticsync_extension_mapping[2]=array("doc",'txt'); # Dokument
$staticsync_extension_mapping[3]=array("mov","3gp","avi","mpg","mp4","flv"); # Video
$staticsync_extension_mapping[4]=array("flv",'mp3','ogg','m4a'); # Audio

The staticsync.php:
$syncdir='/media/archiv/audio/2016'; //
$staticsync_mapped_category_tree = 1;

Output staticsynp.php:

Preloading data... done.
Looking for changes...
Processing Folder: /media/archiv/audio/2016
Processing Folder: /media/archiv/audio/2016/electronic
Not found: electronic @ level 0
Processing file: /media/archiv/audio/2016/electronic/goa.mix.mp3
Collection electronic, theme=Electronic
Processing Folder: /media/archiv/audio/2016/mixes
Processing Folder: /media/archiv/audio/2016/other
Not found: other @ level 0
Processing Folder: /media/archiv/audio/2016/other/sounds
Not found: other~sounds @ level 1
Processing file: /media/archiv/audio/2016/other/sounds/Test.Click.Id3tag.mp3
Collection sounds, theme=Other
Processing file: /media/archiv/audio/2016/other/sounds/Test.Id3tag.mp3
Processing Folder: /media/archiv/audio/2016/other/speech
Not found: other~speech @ level 1
Processing file: /media/archiv/audio/2016/other/speech/Test.HelloWorld.mp3
Collection speech, theme=Other
...done.

After a second run, output looks like:

Preloading data... done.
Looking for changes...
Processing Folder: /media/archiv/audio/2016
Processing Folder: /media/archiv/audio/2016/electronic
Processing Folder: /media/archiv/audio/2016/mixes
Processing Folder: /media/archiv/audio/2016/other
Processing Folder: /media/archiv/audio/2016/other/sounds
Processing Folder: /media/archiv/audio/2016/other/speech
...done.

[Allison M Stec]

You do not want to ingest the files, so set...

$staticsync_ingest=false;

--
ResourceSpace: Open Source Digital Asset Management
http://www.resourcespace.org
---
You received this message because you are subscribed to the Google Groups "ResourceSpace" group.
To unsubscribe from this group and stop receiving emails from it, send an email to resourcespac...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thorsten K]

Sry, I just corrected that now.

Now the file preview is not working anymore, same problem as in the beginning, preview & download is not working

With $staticsync_ingest = false; the imported files look like this in the GUI:
https://drive.google.com/file/d/0BzT0JV89jedRRVdob2ZBNGJjT0U/view?usp=sharing

With $staticsync_ingest = true; it looks:
https://drive.google.com/file/d/0BzT0JV89jedRMnBMNXdfVk95bEk/view?usp=sharing

[Thorsten K]

I found the reason:

The problem was that I just set the $syncdir value in the staticsync.php but not in the config.php. Now it works fine.

Thanks for your help :-)

[scottc]

Hello Allison,

What are other pro's and con's of setting ingest=false? -- I was thinking of setting to false but was under the impression that when uploading files they will still goto the filestore and I did not want to have things in two locations. ingest=true is working for me I just loose the nice folder structure as you indicate.

Thanks Sc

[Allison Stec]

You're right, anything manually uploaded from within the application will be placed in the filestore location regardless of the ingest setting. To keep everything in the same location you'd need to refrain from using the application upload - all uploading would have to be via staticsync.

Ingest or not, you can have staticsync emulate the folder structure as featured collections with $staticsync_folder_structure=true;

What usually happens once people start using RS is that they care less about having a structured 'place' to find a resource. If enough metadata is populated and indexed searching will get them desired results, often much faster than navigating through the emulated folder structure.

I can't think of anything else to mention for pros or cons, but it's been a little while since I've used staticsync heavily.

[Tom Gleason]

One con of ingest=false might be performance issues as the static folder grows. There were some pretty major issues with that which may or may not have since been resolved.

[Allison Stec]

That's a good point.

Using $nogo to skip folders can help with that, but you'd have to know what's been changed/added/removed to speed up sync times.

On 5/5/16 11:03 AM, Tom Gleason wrote:

One con of ingest=false might be performance issues as the static folder grows. There were some pretty major issues with that which may or may not have since been resolved.

--

[scottc]

Thanks for all the feedback !!

$nogo=Folder1 - Would that ignore Folder1 and sub Folders as well? or do you have to each one?

Folder1

   --- Sub FolderA

  -- Sub  FolderB

Thank you,

Sc