compound locks

[Nick Angus]

Has anyone encountered compounds generating lock files when rendering on a renderfarm connected to a workgroup?

It seems to be the same four compounds, although they are all unrelated and from different places.  This has never happened before and has thrown a bit of a spanner in the works!

 

Just thought I would check if anyone has had this happened before.

 

Cheers,Nick

 

 

 

[Stephen Blair]

I've seen that come up from time to time on this list, and in support.

usa.autodesk.com/getdoc/id=TS14206145

[Martin Chatterjee]

Hey Nick,

yeah we've had our share of fun with them... 

Short story:  We've analyzed and reported the issue, it's sort of "explainable" and reproducible to a certain extent. But I wouldn't hold my breath for getting a fast fix for this as it... 

So we adapted our renderfarm manager to find the offending blades and restart the renderjob on these ones automatically. Typically the lock files disappear when the corresponding offending xsibatch process is not alive anymore. So bottom line this issue "costs" us only roughly 3-5 minutes of downtime on a handful of blades which is a non-issue in relation to our overall render times. But it still feels a bit dodgy and of course I'd rather have it resolved.

Long story:  This is what happens as far as I understand it:

1.) On startup the contents of every linked workgroup get parsed by every machine in order to correctly identify and register all plugins/shaders/compounds/...

2.) For .xsicompound and .xsirtcompound files (and only for those, God knows why) the parsing machine will use a locking mechanism:

             - for foo.xsicompound  create a lock file named foo.xsicompound.lock (after first making sure that this file does not already exist)

             - parse foo.xsicompound

             - delete the lock file

3.) When loads of machines do this at pretty much the same time the chance rises for collisions. Also if your file server is not very good with handling a lot of requests for interaction with lots of TINY files this will also dramatically rise the chance for collisions

Also I'd love to hear the reason for this whole lock-file thingy. I just don't get it - this should just be text-file read access, just the way ALL other non-compiled workgroup items are alreay handled... 

Hope that helps a bit...

Cheers, Martin

--
       Martin Chatterjee
 
[ Freelance Technical Director ]
[   http://www.chatterjee.de   ]

[ https://vimeo.com/chatterjee ] 

[Nick Angus]

Thanks Stephen, nice to still have you around!

 

N

[Nick Angus]

Thanks Martin, for taking the time to answer so thoroughly, an obvious sign of your frustration too!

I am surprised it has never come up before for us, I get the errors in the renderlogs, but have never had it create and actual lock file before!

I will keep you informed of my progress and of any findings…

 

Cheers, Nick

 

From: softimag...@listproc.autodesk.com [mailto:softimag...@listproc.autodesk.com] On Behalf Of Martin Chatterjee
Sent: Friday, 2 November 2012 8:29 PM
To: soft...@listproc.autodesk.com
Subject: Re: compound locks

 

Hey Nick,