Verifying NFS hard links are working?

[Davis, Justin W]

Hello All.

Is there an easy way to verify that hard links are working as needed on a shared NFS volume? Is there any consensus as to whether NFSv3 or NFSv4 is better suited?

I'm trying to troubleshoot why my "workspace" folder is unnervingly large (1.5TB in the workspace folder out of 2.4TB accrued over the past 3 weeks).

Thank you kindly,
Justin

--
Justin Davis
Liberal Arts ITS
Sr Systems Administrator
justi...@mail.utexas.edu

[Jan Koppe]

Hey Justin,

a very quick way would be to run ls -l inside the opencast data folders.
The second column should show you the number of hardlinks for this file.
Alternatively you could list inodes with ls -i, and then search for them
recursively using find . -inum $INUM

Not sure if this will work over NFS as well (though it probably should?
need to read up on that), if in doubt run on the storage server itself.

Regards,
Jan

--
Jan Koppe
eLectures / LearnWeb
Westfälische Wilhelms-Universität
Georgskommende 25 - Room 310
48143 Münster/Westf. - Germany
Tel. + 49 (0) 251 - 83 29295
E-mail: jan....@wwu.de

[Greg Logan]

Hi Justin, 

Jan's idea is a good one, as is verifying that your workflows have the cleanup step at the end. If that is not there then the workspace directory won't get cleaned up at the end of the workflow! 

And as a heads up, I'm not sure which version you are running but the workspace is generally considered a temporary directory as well - the inputs ought to be in the archive and the outputs in files. It might be safe to remove those files if all of your current workflows are finished.

G

G

--
You received this message because you are subscribed to the Google Groups "Opencast Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to users+un...@opencast.org.

[Lars Kiesow]

Hi,
@Justin: ensure that this is set correctly, as only if this is set to
the same value on all servers, Opencast will use a shared storage which
is a requirement for using hard linking:
org.opencastproject.file.repo.url=${org.opencastproject.admin.ui.url}
It's also specified in the multi-server-setup-guide:
https://docs.opencast.org/develop/admin/installation/multiple-servers/#step-5-configure-opencast

@Greg: There are some bugs in the usage of workspace and working file
repository which can easily cause problems if things are slightly
mis-configured

@all: Maybe it would make sense to simply add a log entry for if hard
linking is enabled. Even if it's only a debug log, it would make
verifying easier.

Regards,
Lars

[Greg Logan]

Lars, last I checked there was a log entry for that at INFO level already.

Justin, I'm assuming that log entry is there? It pops up on Opencast startup. 

G

[Davis, Justin W]

Hi Greg & Lars - Yes, actually, there is a clear message at server start:

INFO | (WorkspaceImpl:209) - Hard links between the working file repository and the workspace enabled

Thank you for pointing me in the right direction. I'm still cleaning up a ton of problematic ingests (Opencast was initially misconfigured with an opencast-allinone and several opencast-worker nodes). I suspect that's largely why my workspace area is taking up approximately 50% of the total storage used (1.2TB/2.4).

Thanks again!
- Justin

--
Justin Davis
Liberal Arts ITS
Sr Systems Administrator
justi...@mail.utexas.edu