icecache

[adrian wyer]

Hey guys, im trying to render a 4000 frame emfluid icecache using emrpc (gave up on volumes, it was going to take 3000 hours!)

I have an icetree that reads the cache, and then renders a frame using the emrpc compound….. so far so good

 

the cache files are fairly big (about 90Mb each)

After about 100 frames (when the particle count gets pretty high) it is taking more and more time to load each frame (and more memory)

 

Now this scene managed to write the cache over a day (or two) without any probs, but reading it back in seems to take an exponentially increasing amount of time

Has anyone seen this, and more importantly, found a way around it?!

I replaced the current frame node with a scalar and set it to 100 and it took ages to load that frame…. But if I leave the current frame node attached, and jump to frame 100 it takes hours!

 

I thought about setting up several scenes, and replacing the current frame node with a keyed scalar, so each scene would load/render 200 frames or so…. But this seems like a lot of dicking around……

 

A little help?

 

 

Thanks

 

a

 

Adrian Wyer
Fluid Pictures
16-18 Beak St.
London W1F 9RD
T.++44(0) 207 434 3565
F.++44(0) 207 100 1989
E. adria...@fluid-pictures.com
www.fluid-pictures.com

Fluid Pictures Limited is registered in England and Wales.
Company number:5657815
VAT number: 872 6893 71

 

[Ciaran Moloney]

Hi,
First off - make sure that the ICE tree reading the cache is in the modeling region (or you could probably also change the Environment's simulation play mode to 'interactive').

If you already have that in place, you might want to think about caching locally, if the cache files are absurdly large (but 90Mb sounds fine).

Ciaran

[Robert Chapman]

yeah ive done this the way you describe - maybe not as many particles at once as you are attempting - about 5 or 6 million as far as I can remember  - have you tried the particle display property set to points and a really low display % like 1 or 2 % ? 

[Robert Chapman]

heheh just realised your crappy ati card prolly cant display all the particles...      but yeah seriously , I wouldnt ask the xsi display port to display 100% of millions of particles it will be slow regardless of the card you are using.    

[Ciaran Moloney]

I'd reckon loading the cache is the bottle neck. I don't think XSI is smart enough to read only displayed particles from the cache and probably loads the whole thing first. Smells like a feature request...

[adrian wyer]

Will give the display thing a try…. But this is on nvidia card anyway…..  the slowdown is during loading, not display, and the files are local…….

 

Will also try having the icetree under modelling as Ciaran suggested

 

a

 

Adrian Wyer
Fluid Pictures
16-18 Beak St.
London W1F 9RD
T.++44(0) 207 434 3565
F.++44(0) 207 100 1989
E. adria...@fluid-pictures.com
www.fluid-pictures.com

Fluid Pictures Limited is registered in England and Wales.
Company number:5657815
VAT number: 872 6893 71

[adrian wyer]

Right, when I jump to a frame in the timeline, you would THINK that soft is smart enough to ONLY load that icecache frame, not count through all of the one upto that point!!!

 

This is a serious flaw, and is evidenced whenever you have a scene with a cached ICE cloud, if you jump it will often say Re-simulating even when the simulation part of the tree is unplugged… I just assumed this really meant “loading each frame until the current one, and discarding each frame until the current one”!

 

a

 

Adrian Wyer
Fluid Pictures
16-18 Beak St.
London W1F 9RD
T.++44(0) 207 434 3565
F.++44(0) 207 100 1989
E. adria...@fluid-pictures.com
www.fluid-pictures.com

Fluid Pictures Limited is registered in England and Wales.
Company number:5657815
VAT number: 872 6893 71

From: softimag...@listproc.autodesk.com [mailto:softimag...@listproc.autodesk.com] On Behalf Of Ciaran Moloney
Sent: 12 August 2010 12:35
To: soft...@listproc.autodesk.com
Subject: Re: icecache

 

I'd reckon loading the cache is the bottle neck. I don't think XSI is smart enough to read only displayed particles from the cache and probably loads the whole thing first. Smells like a feature request...

[Robert Chapman]

if you change to interactive mode like Ciaran suggested it doesnt do that...

[adrian wyer]

Sweet

 

a

 

Adrian Wyer
Fluid Pictures
16-18 Beak St.
London W1F 9RD
T.++44(0) 207 434 3565
F.++44(0) 207 100 1989
E. adria...@fluid-pictures.com
www.fluid-pictures.com

Fluid Pictures Limited is registered in England and Wales.
Company number:5657815
VAT number: 872 6893 71

[John Richard Sanchez]

Have you tried caching and then reading with the new cache manager instead of caching with the Ice Tree?

--
John Richard Sanchez
www.johnrichardsanchez.com

[Ciaran Moloney]

I think it's the correct behaviour. After all the tree is in the simulate region, so XSI is expecting to do some simulation on it. So what if the simulate particles node is unplugged? You might have your own simulation node there or something...

[John Richard Sanchez]

I have been cacheing ice clouds with the cache manager and I dont get simulating particles when I scrub the timeline. However; I am not caching millions of particles either.:)

[Yanick Pomerleau]

Hi Adian,

It's not suppose to read all the frame. It's supposed to read only the current frame.
What kind of caching do you use? ICE Cache On File node or the Cache Manager Simulation time control PPG or the Cache Manager Animation PPG?

Yanick Pomerleau
Software Developer
Autodesk Media and Entertainment

Autodesk, Inc.
10 Duke
Montreal, QC H3C 2L7

Direct 514 954-7245

[cid:image0...@01CB3A26.079C6790]

From: softimag...@listproc.autodesk.com [mailto:softimag...@listproc.autodesk.com] On Behalf Of adrian wyer
Sent: August-12-10 7:48 AM
To: soft...@listproc.autodesk.com
Subject: RE: icecache

Right, when I jump to a frame in the timeline, you would THINK that soft is smart enough to ONLY load that icecache frame, not count through all of the one upto that point!!!

This is a serious flaw, and is evidenced whenever you have a scene with a cached ICE cloud, if you jump it will often say Re-simulating even when the simulation part of the tree is unplugged... I just assumed this really meant "loading each frame until the current one, and discarding each frame until the current one"!

a

Adrian Wyer
Fluid Pictures
16-18 Beak St.
London W1F 9RD
T.++44(0) 207 434 3565
F.++44(0) 207 100 1989

E. adria...@fluid-pictures.com<blocked::mailto:adria...@fluid-pictures.com>
www.fluid-pictures.com<blocked::http://www.fluid-pictures.com/>

Fluid Pictures Limited is registered in England and Wales.
Company number:5657815
VAT number: 872 6893 71
________________________________
From: softimag...@listproc.autodesk.com [mailto:softimag...@listproc.autodesk.com] On Behalf Of Ciaran Moloney
Sent: 12 August 2010 12:35
To: soft...@listproc.autodesk.com
Subject: Re: icecache

I'd reckon loading the cache is the bottle neck. I don't think XSI is smart enough to read only displayed particles from the cache and probably loads the whole thing first. Smells like a feature request...

On Thu, Aug 12, 2010 at 12:32 PM, Robert Chapman <tekan...@gmail.com<mailto:tekan...@gmail.com>> wrote:
heheh just realised your crappy ati card prolly cant display all the particles... but yeah seriously , I wouldnt ask the xsi display port to display 100% of millions of particles it will be slow regardless of the card you are using.

On 12 August 2010 12:19, Robert Chapman <tekan...@gmail.com<mailto:tekan...@gmail.com>> wrote:
yeah ive done this the way you describe - maybe not as many particles at once as you are attempting - about 5 or 6 million as far as I can remember - have you tried the particle display property set to points and a really low display % like 1 or 2 % ?

On 12 August 2010 12:10, Ciaran Moloney <moloney...@gmail.com<mailto:moloney...@gmail.com>> wrote:
Hi,
First off - make sure that the ICE tree reading the cache is in the modeling region (or you could probably also change the Environment's simulation play mode to 'interactive').

If you already have that in place, you might want to think about caching locally, if the cache files are absurdly large (but 90Mb sounds fine).

Ciaran

On Thu, Aug 12, 2010 at 12:04 PM, adrian wyer <adria...@fluid-pictures.com<mailto:adria...@fluid-pictures.com>> wrote:
Hey guys, im trying to render a 4000 frame emfluid icecache using emrpc (gave up on volumes, it was going to take 3000 hours!)

I have an icetree that reads the cache, and then renders a frame using the emrpc compound..... so far so good

the cache files are fairly big (about 90Mb each)
After about 100 frames (when the particle count gets pretty high) it is taking more and more time to load each frame (and more memory)

Now this scene managed to write the cache over a day (or two) without any probs, but reading it back in seems to take an exponentially increasing amount of time
Has anyone seen this, and more importantly, found a way around it?!

I replaced the current frame node with a scalar and set it to 100 and it took ages to load that frame.... But if I leave the current frame node attached, and jump to frame 100 it takes hours!

I thought about setting up several scenes, and replacing the current frame node with a keyed scalar, so each scene would load/render 200 frames or so.... But this seems like a lot of dicking around......

[adrian wyer]

It was the cache on file node in the ICE tree.....

However I was reading back in the same tree (just disconnecting the
simulation parts of the tree and leaving the read cache node

When I set up a new cloud, with an Ice tree under the modelling stack, put
the read cache node on that and set the environment to interactive it worked
fine

A

Adrian Wyer
Fluid Pictures
16-18 Beak St.
London W1F 9RD
T.++44(0) 207 434 3565
F.++44(0) 207 100 1989
E. adria...@fluid-pictures.com
www.fluid-pictures.com

Fluid Pictures Limited is registered in England and Wales.
Company number:5657815
VAT number: 872 6893 71

[Sandy Sutherland]

I always use the Cache manager - to write caches and read them - seems
to work pretty well - and also rendering fine on our Linux Farm too!!!

S.

[christian keller]

it�s always clever to read it back on a new cloud. keeps things clean ...
with the import/export thing you have a nice workflow.
sim-export and then switch to your rendering pass with the already
imported mixer cloud and render. as long as you overwrite the sim
it�ll reflect your changes. you rarely need the ice cachenode, unless
you want to do something in a simulation region .

christian keller
visual effects|direction

+49 0179 69 36 248
chr...@gmx.de