Improve Search Speed

[Marc]

I have the latest version of ResourceSpace installed on an Ubuntu 22.04 VM in Unraid. The VM is running on an SSD and configured with 16GB RAM and uses 8 CPU cores. The photos are actually stored on an array consisting of several 7200 RPM drives.

Over the last year the search function has slowed where it now takes 10-15 sec. to load results as well as the thumbnails at the bottom of the page to populate. RS was configured to use MySQL within the VM but as an experiment I did install MariaDB in a docker container. That improved the search speed by 1-2 sec.

Doing a Performance test in RS shows MySQL throughput is 1092.9, CPU benchmark is 470.7, and Disk write speed is 65.9. Total assets in the collection is 71,613.

Does anyone have suggestions to increase performance of searches as 10+ sec. seems excessive.

Thank you.

[Marc]

It turns out it was because the resized files were stored on the array of several mechanical drives. As soon as I moved the resized files to an SSD and created a symlink searches went from 12-15 sec to 1-2 sec.

[Hazem Alashi]

Hi Marc

We could also benefit from this fast searches, could you elaborate more on that?

Or where to find more information please?

Thank you

--
ResourceSpace: Open Source Digital Asset Management
http://www.resourcespace.com
---
You received this message because you are subscribed to the Google Groups "ResourceSpace" group.
To unsubscribe from this group and stop receiving emails from it, send an email to resourcespac...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/resourcespace/ce1b1b78-3eee-4301-b53e-f7d01cc81104n%40googlegroups.com.

[Marc]

Sure.

All my assets (photos/video) are on an array of mechanical drives. These drives, at least through my NAS (Unraid), tend be slower as its reading/writing all drives. This seems to be what caused the search slowdown as it was looking for all the thumbnails/previews spread out on these drives. The solution was to move these files to an SSD.

So I created a resized directory on the SSD and used rsync -a to move all the files from the filestore/resized directory to the new directory on the SSD. I then renamed the filestore/resized directory to resized_backup (as a precaution) and created a new resized directory. I then mounted the SSD resized directory to the new resized directory in filestore in fstab. Now when ResourceSpace looks for the thumbnails/previews it access them on the SSD.

Hopefully that makes sense. If you need it broken down a bit more let me know.

[Kevin Chung]

Marc 

Would you happen to know about file store and uploading permissions ? 

I am getting Filename invalid error. When i attempt to upload a file 

Best,

Kevin Chung

Director of IT

Children's Health Fund

475 Riverside Dr, Ste. 630

New York, NY  10027

kch...@chfund.org

O:212-535-9400 x8671

To view this discussion, visit https://groups.google.com/d/msgid/resourcespace/6ca80992-852d-43b3-a4fa-998896f0001bn%40googlegroups.com.

[Marc]

What are you running ResourceSpace on? VM? If so, is it in an NAS? Is it a samba share or something g else? Permissions with shared directories have always been a pain the butt for me and takes a lot of trial and error.

[Kevin Chung]

The original server is a on physical box

The new server i opted to host on a Vm in virtualbox

I was able to backup and restore the database

see all the files and collections

old usernames worked

new url

all that works

Now when it's time to upload a new file on the new server I get filename invalid error. 

Yes the filestore is a symbolic link to a NAS share. So the only part of the restore process I did not follow completely

They suggested copying the filestore, I opted to just recreate the symbolic link on the new server as copying the filestore back to the NAS did not add up. 

Any suggestions on what to do?

Best,

Kevin Chung

Director of IT

Children's Health Fund

475 Riverside Dr, Ste. 630

New York, NY  10027

kch...@chfund.org

O:212-535-9400 x8671

To view this discussion, visit https://groups.google.com/d/msgid/resourcespace/47e2ac3d-d7ae-49ab-bf55-57dee29b90e4n%40googlegroups.com.

[Marc]

Like I said, permissions are always challenging for me, and it involves a lot of trial and error. So, you have ResourceSpace installed in a VM through Virtualbox. What is the OS of the box where the VM resides? How are mounting filestore? Is it an SMB share or something else?

With Unraid, sharing using SMB wasn't working because I wanted those folders secured with a username and password and doing so prevented ResourceSpace from writing. I could never figure out why. On the suggestion from a friend, I elected to use 9p since that seemed more compatible with Unraid in this instance.

Here is my configuration:

NAS has three shared directories: filestore, resized, and rs_system (i.e. system). Filestore resides on the array of mechanical drives totaling 54TB. Resized and rs_system reside on an SSD.

Ubuntu VM: I created an unraid directory in root. No specific reason, just to help keep things organized in my head. I then created a symlink where ResourceSpace is installed for filestore pointing to /unraid. (ln -s /unraid filestore)  I then mounted the NAS filestore to /unraid, resized to /unraid/resized, and rs_system to /unraid/system.

Here is what my fstab configuration looks like. Yours will probably be different since mine is for Unraid.

------

filestore /unraid 9p  user=<username>,password=<password>,trans=virtio,version=9p2000.L,rw,_netdev 0 2
resized /unraid/resized 9p user=<username>,password=<password>,trans=virtio,version=9p2000.L,rw,_netdev 0 2
rs_system /unraid/system 9p  user=<username>,password=<password>,trans=virtio,version=9p2000.L,rw,_netdev 0 2

------

I'm not sure if you've done this or if it does anything, but try adding the $storagedir and $storageurl to your ResourceSpace config.

------

$storagedir = '/var/www/html/filestore';
$storageurl = 'http://<url>/unraid';

------

Make sure everything in filestore has the correct user:group permissions. Mine is elec:users but yours may be different.

Hopefully this helps point you in the right direction.