Developers (developers, developers, ...)
Davide Pasca
I've been writing some sort of renderer in my spare time for the last
year or so.
It's far from being RenderMan compliant, but that's the general goal
( http://kazzuya.com/ribtools ).
I'm a complete beginner when it comes to off-line rendering. And it's
been a quite a challenge to get this far 8)
I don't see much activity around here, but I suspect that the old-
schoolers are probably still checking the group. So, I'll try to
ask: ...is there a currently more active and developer-inclined
discussion forum ?
I surely could use some hints 8)
Thank you,
Davide
Olivier Paquet
think of would be the aqsis and pixie forums but you probably already
know about those. You might also get some hints from people
programming with the RenderMan API (instead of implementing it) and
this group isn't such a bad place for that. Or are you after more
generic programming advice? Something in-between (rendering related
but not RenderMan specific) ?
Olivier
Davide Pasca
On Dec 3, 1:45 am, Olivier Paquet <Olivier3...@hotmail.com> wrote:
> You mean a RenderMan renderer developer forum? The closest thing I can
> think of would be the aqsis and pixie forums but you probably already
> know about those. You might also get some hints from people
I know Pixie and Aqsis and I think I'll follow your suggestion and try
follow the relative developer sections of the forums.
However, I was hoping for something a bit more neutral, not focussed
on a specific product.
For example, for ray-tracing, http://ompf.org/forum is quite active.
But I guess that REYES isn't as popular 8)
> programming with the RenderMan API (instead of implementing it) and
> this group isn't such a bad place for that. Or are you after more
> generic programming advice? Something in-between (rendering related
> but not RenderMan specific) ?
I'm experimenting writing a RenderMan-like renderer, so, while reading
on API usage is very useful, my main questions are more about the
internals of a REYES-style renderer.
I found the book "Production Rendering" (edited by Ian Stephenson)
very useful, but it's more a of starting point.
I wonder how many people bought the book and how many of those tried
to put it in practice 8)
Anyhow, thank you for the suggestions.
Davide
Olivier Paquet
> However, I was hoping for something a bit more neutral, not focussed
> on a specific product.
That will likely be difficult. Most of us who know about REYES do so
because we're working or have worked on a specific product. Unlike ray
tracing, there doesn't seem to be much academic research about REYES.
That's a bit unfortunate as REYES today is quite different from the
original papers about it. As far as I can tell, everyone came up with
their own solutions (or not) to the various problems.
> For example, for ray-tracing,http://ompf.org/forumis quite active.
> But I guess that REYES isn't as popular 8)
Yes, I don't think it is used much outside the film industry. Its
inherent complexity also makes it harder to discuss and creates a
large barrier of entry for anyone wanting to experiment with it. You
can write a triangle ray tracer in an afternoon... not so for even a
very basic REYES engine (well, I don't think so anyway). It also
doesn't split in interesting (outside the scope of rendering) and
completely separate problems like a ray tracer (ie. space
partitioning, bounding volume intersection tests, primitive
intersection tests). That probably contributes to the lack of
research.
> I'm experimenting writing a RenderMan-like renderer, so, while reading
> on API usage is very useful, my main questions are more about the
> internals of a REYES-style renderer.
Well feel free to ask. Some stuff I can answer and some I can't.
> I found the book "Production Rendering" (edited by Ian Stephenson)
> very useful, but it's more a of starting point.
> I wonder how many people bought the book and how many of those tried
> to put it in practice 8)
I imagined as much, which is why I haven't bought it yet. I probably
will at some point, mostly out of curiosity. Anyway, good luck in your
quest!
Olivier
Davide Pasca
> On 10 déc, 05:32, Davide Pasca <dpa...@gmail.com> wrote:
>
> > However, I was hoping for something a bit more neutral, not focussed
> > on a specific product.
>
> That will likely be difficult. Most of us who know about REYES do so
> because we're working or have worked on a specific product. Unlike ray
> tracing, there doesn't seem to be much academic research about REYES.
> That's a bit unfortunate as REYES today is quite different from the
> original papers about it. As far as I can tell, everyone came up with
> their own solutions (or not) to the various problems.
..I guess part of the lack of interest is that GPU is were most of the
buzz is and there is relatively little work on getting REYES on a GPU
(I suspect that most are waiting for GPUs to get flexible enough, I
know I am 8)
> large barrier of entry for anyone wanting to experiment with it. You
> can write a triangle ray tracer in an afternoon... not so for even a
> very basic REYES engine (well, I don't think so anyway). It also
> doesn't split in interesting (outside the scope of rendering) and
> completely separate problems like a ray tracer (ie. space
> partitioning, bounding volume intersection tests, primitive
> intersection tests). That probably contributes to the lack of
> research.
I agree. It's true that some game devs are starting to talk about
REYES (thus making it more popular outside the films industry), but
that's all generalized as "micropolygons".
Every time someone asks me about my "micropolygon renderer" I pretend
I don't understand.. it sounds almost as bad as "real-time
raytracing"... scary generalizations.
Getting the RMan state machine right and an half-working RSL compiler
alone is quite a task. Key to making a usable renderer, but not
necessarily requiring micropolys.
> Well feel free to ask. Some stuff I can answer and some I can't.
Great ! Then I'll be posting some questions in this newsgroup 8)
> > I found the book "Production Rendering" (edited by Ian Stephenson)
> > very useful, but it's more a of starting point.
[...]
> I imagined as much, which is why I haven't bought it yet. I probably
> will at some point, mostly out of curiosity. Anyway, good luck in your
> quest!
If you are already in the renderers development business, then I'm not
sure how much the book can tell you. For me it was really invaluable.
Thank you for the hints.. 8)
Davide
Olivier Paquet
> ..I guess part of the lack of interest is that GPU is were most of the
> buzz is and there is relatively little work on getting REYES on a GPU
> (I suspect that most are waiting for GPUs to get flexible enough, I
> know I am 8)
The core of REYES is a rather bad fit for a GPU as far as I can tell.
It's a fundamentally serial algorithm with only some tasks being
easily done in parallel. The shading will be doable at some point but
there will remain some big issues:
1) You can't put everything on the GPU. Some shadeops (raytracing in
particular) just don't fit for realistic use.
2) Render farms don't have discrete, powerful GPUs, for several
reasons.
3) People constantly want more flexibility for everything. This
clashes with the requirements of the GPU.
You could also get some speedup with the sampling but Gelato showed it
isn't all that impressive.
> I agree. It's true that some game devs are starting to talk about
> REYES (thus making it more popular outside the films industry), but
> that's all generalized as "micropolygons".
Don't get me started on game devs and their messed up nomenclature :)
> Every time someone asks me about my "micropolygon renderer" I pretend
> I don't understand.. it sounds almost as bad as "real-time
> raytracing"... scary generalizations.
Hehe at least they're asking!
> Getting the RMan state machine right and an half-working RSL compiler
> alone is quite a task. Key to making a usable renderer, but not
> necessarily requiring micropolys.
Indeed: bmrt. And RSL is a major reason RenderMan has remained
relevant for so long. It's also why a lot of claims of "real time
raytracing" are complete crap. Hardcoded plastic shaders are not
realistic.
> > > I found the book "Production Rendering" (edited by Ian Stephenson)
> > > very useful, but it's more a of starting point.
> [...]
> > I imagined as much, which is why I haven't bought it yet. I probably
> > will at some point, mostly out of curiosity. Anyway, good luck in your
> > quest!
>
> If you are already in the renderers development business, then I'm not
> sure how much the book can tell you. For me it was really invaluable.
Well except a few scary dark corners, I know 3Delight inside and out
so I guess not very much indeed. It'd be mostly curiosity about how
others perceived the various problems. From the contents it does looks
like a decent starting point to get a renderer off the ground.
> Thank you for the hints.. 8)
You're welcome.
Olivier
Jonathas
> The core of REYES is a rather bad fit for a GPU as far as I can tell.
> It's a fundamentally serial algorithm with only some tasks being
> easily done in parallel. The shading will be doable at some point but
> there will remain some big issues:
I high recommend you to take a look in Renderants: http://www.kunzhou.net/#renderants
Zhou has made a impressive work. It isn't a complete renderman
implementation (ray tracing, global illumination, and others things
are out), but the entire ''classic'' pipeline is in the GPU. The
results, in speed and image quality, are very good.
Jonathas.
Olivier Paquet
> I high recommend you to take a look in Renderants:http://www.kunzhou.net/#renderants
> Zhou has made a impressive work. It isn't a complete renderman
> implementation (ray tracing, global illumination, and others things
> are out), but the entire ''classic'' pipeline is in the GPU. The
> results, in speed and image quality, are very good.
Not quite the whole thing yet. As far as I understood it (I only had
time to give the paper a quick read), there is no depth culling in
their algorithm. This is not a trivial issue as culling introduces a
dependency on sampling (nearly the last stage) in the bucketing and
splitting code (the very first stage). This could severely mess with
the nice parallelism. It can certainly be dealt with but it's not
clear how efficient it would be. It seems like it would be a tradeoff
between good culling and good parallelism.
Still, they apparently put an awful lot of work into writing a large
amount of GPU code and the results are fairly impressive. The shader
compiler must be some interesting beast given what it needs to do. But
I have some doubts whether the algorithm can be made to scale.
Olivier
Davide Pasca
> 2) Render farms don't have discrete, powerful GPUs, for several
> reasons.
Clearly NVidia is moving towards this direction though. I remember a
presentation from a famous production company that uses GPUs for
simulation.
Ray tracing is for sure one of the goals, with NVidia itself already
with a couple of APIs for it and owning Mental Images..
> 3) People constantly want more flexibility for everything. This
> clashes with the requirements of the GPU.
I agree, however GPUs are becoming more and more general purpose.
> You could also get some speedup with the sampling but Gelato showed it
> isn't all that impressive.
Gelato is pretty old and I suspect not very much supported. As far as
I understand, the original authors aren't with NVidia anymore.
> > Every time someone asks me about my "micropolygon renderer" I pretend
> > I don't understand.. it sounds almost as bad as "real-time
> > raytracing"... scary generalizations.
>
> Hehe at least they're asking!
Someday I have to sit and prepare a good answer that will work on my
gamedev-oriented colleagues and bosses 8)
> Indeed: bmrt. And RSL is a major reason RenderMan has remained
> relevant for so long. It's also why a lot of claims of "real time
> raytracing" are complete crap. Hardcoded plastic shaders are not
> realistic.
eheheh.. eventually the best application for all the real-time ray-
tracing research will be non-real-time ray-tracing 8)
> Well except a few scary dark corners, I know 3Delight inside and out
> so I guess not very much indeed. It'd be mostly curiosity about how
> others perceived the various problems. From the contents it does looks
> like a decent starting point to get a renderer off the ground.
Some things are right under your nose and click right away when
reading (from a game developer perspective). I definitely suggest the
book to those that are in the real-time field (those that probably
don't read this newsgroup though 8)
I read little of the GI chapter.. that section seems more advanced and
it touches photon mapping, irradiance caching and about caching
displaced geometry.
There is also a pretty detailed explanation about making a good noise
function and why it is important.
poof
Davide
Davide Pasca
> On 16 déc, 06:42, Jonathas <jon.co...@gmail.com> wrote:
>
> > I high recommend you to take a look in Renderants:http://www.kunzhou.net/#renderants
> > Zhou has made a impressive work. It isn't a complete renderman
> > implementation (ray tracing, global illumination, and others things
> > are out), but the entire ''classic'' pipeline is in the GPU. The
> > results, in speed and image quality, are very good.
>
> Not quite the whole thing yet. As far as I understood it (I only had
> time to give the paper a quick read), there is no depth culling in
Actually, I was at Siggraph Asia today where one of the authors
presented RenderAnts.
Unfortunately I arrived late when the presentation was already
started.
At the end, one person asked about coping with cracks introduced in
the splitting stage..
And the answer was that they are currently not dealing with cracks
(this is also written in the paper). And the cracks are often hidden
by the high (13x13) sub-sampling.
Another person asked about how many of the shading opcodes have been
implemented and suggested on the complexity of that..
But the answer somehow ended up being on some performance figures,
basically saying that their compiler produces code that uses less
registers than Pixie.
Anyhow, the goal of RenderAnts is to show that implementing the whole
Reyes pipeline is possible.. and I think that it's pretty impressive.
Performance scalability is good, but not exceptional, at 2x for 3
GPUs. Network distributed rendering currently not supported and in
general not really trying to compete with the set of features of
PRMan.
Code is apparently available to other researches (universities only ?)
that request it.
baubau
Davide
Olivier Paquet
> > 2) Render farms don't have discrete, powerful GPUs, for several
> > reasons.
>
> Clearly NVidia is moving towards this direction though. I remember a
> presentation from a famous production company that uses GPUs for
> simulation.
> Ray tracing is for sure one of the goals, with NVidia itself already
> with a couple of APIs for it and owning Mental Images..
In render farms? How so? Last I checked, GPUs are getting bigger,
hotter, more expensive and ready to be teamed up in pairs. Not exactly
compatible with a render farm.
> > 3) People constantly want more flexibility for everything. This
> > clashes with the requirements of the GPU.
>
> I agree, however GPUs are becoming more and more general purpose.
Which is why they will merge back into the CPU and nvidia will likely
get out of the PC business (or go out of business completely). The
writing has been on the wall for a few years already. The history of
PCs is littered with the remains of makers of specialized chips who
were once king of the hill (e.g. think about laughable the idea of a
100$ "sound card" is to most people today).
Olivier
Davide Pasca
> On Dec 19 2009, 8:33 am, Davide Pasca <dpa...@gmail.com> wrote:
>
> > > 2) Render farms don't have discrete, powerful GPUs, for several
> > > reasons.
>
> > Clearly NVidia is moving towards this direction though. I remember a
> > presentation from a famous production company that uses GPUs for
> > simulation.
> > Ray tracing is for sure one of the goals, with NVidia itself already
> > with a couple of APIs for it and owning Mental Images..
>
> In render farms? How so? Last I checked, GPUs are getting bigger,
> hotter, more expensive and ready to be teamed up in pairs. Not exactly
> compatible with a render farm.
Well NVidia is proposing RealityServer ( http://www.nvidia.com/object/realityserver.html
)
I'm not sure how much following is going to have, but I wouldn't count
it out just yet.
About the simulation using GPU in production (but not farms). I was
referring to LucasFilm presentation at GTC09.
Here is a clip about that: http://www.youtube.com/watch?v=3eCk3316xdA
> > I agree, however GPUs are becoming more and more general purpose.
>
> Which is why they will merge back into the CPU and nvidia will likely
> get out of the PC business (or go out of business completely). The
> writing has been on the wall for a few years already. The history of
> PCs is littered with the remains of makers of specialized chips who
> were once king of the hill (e.g. think about laughable the idea of a
> 100$ "sound card" is to most people today).
Sound and graphics are obviously two very different things. Visual
related rendering and simulation is quite a bit more expensive and
also receives a lot more attention.
I can see how GPUs are painful to implement proper renderers.. but if
we had to judge from the Intel's recently postponed attempt to nuke
GPUs.. I'm not sure that NVidia is going to go away so quickly.
8)
Davide
Olivier Paquet
> At the end, one person asked about coping with cracks introduced in
> the splitting stage..
>
> And the answer was that they are currently not dealing with cracks
> (this is also written in the paper). And the cracks are often hidden
> by the high (13x13) sub-sampling.
That's a fairly common misunderstanding of the problem :( Cracks (99%
of them anyway) are not hidden by oversampling. They are merely
antialiased better and are still very much visible in many cases. But
that's to be expected if they haven't looked at the problem.
> Another person asked about how many of the shading opcodes have been
> implemented and suggested on the complexity of that..
>
> But the answer somehow ended up being on some performance figures,
> basically saying that their compiler produces code that uses less
> registers than Pixie.
Weird answer indeed :) But the number of opcodes is not really
relevant if you have enough resources to implement them as they are
mostly independant. And I figure Pixie's compiler doesn't care that
much about registers because it doesn't really have to.
> GPUs. Network distributed rendering currently not supported and in
> general not really trying to compete with the set of features of
> PRMan.
That would be quite crazy anyway given the amount of code they must
have poured into that soup ;-)
Olivier
Olivier Paquet
> Well NVidia is proposing RealityServer (http://www.nvidia.com/object/realityserver.html
> )
> I'm not sure how much following is going to have, but I wouldn't count
> it out just yet.
I got a buzzword buffer overflow trying to read that page :) Looks
like they're trying to create a new market for their products, which
is understandable in their current position. But I doubt it will be
price competitive with generic server hardware for general rendering
and the other tasks usually sent to a render farm.
> About the simulation using GPU in production (but not farms). I was
> referring to LucasFilm presentation at GTC09.
Oh I somehow missed "simulation" in my first read of your post. For
simulation yes a GPU makes more sense as it is a more limited and
better defined problem. AFAIK it also involves a lot of number
crunching with large uniform datasets that GPUs can be good at.
> > > I agree, however GPUs are becoming more and more general purpose.
>
> > Which is why they will merge back into the CPU and nvidia will likely
> > get out of the PC business (or go out of business completely). The
> > writing has been on the wall for a few years already. The history of
> > PCs is littered with the remains of makers of specialized chips who
> > were once king of the hill (e.g. think about laughable the idea of a
> > 100$ "sound card" is to most people today).
>
> Sound and graphics are obviously two very different things. Visual
> related rendering and simulation is quite a bit more expensive and
> also receives a lot more attention.
So it's a more complex problem to code a solution for and that
solution needs more FLOPS and bandwidth. That's not an argument for
discrete chips. It's simply an argument for more computing power.
Orders or magnitude more than for sound, I'll admit, but computing
power has increased by orders of magnitude in the last 20 years. So
eventually, while it may still be technically possible to build a
faster GPU, it won't make economic sense vs an integrated solution for
most uses.
> I can see how GPUs are painful to implement proper renderers.. but if
> we had to judge from the Intel's recently postponed attempt to nuke
> GPUs.. I'm not sure that NVidia is going to go away so quickly.
I guess we'll have to wait and see. I don't expect it to take more
than 5-10 years.
Olivier
Davide Pasca
> On Jan 6, 6:23 am, Davide Pasca <dpa...@gmail.com> wrote:
> So it's a more complex problem to code a solution for and that
> solution needs more FLOPS and bandwidth. That's not an argument for
> discrete chips. It's simply an argument for more computing power.
> Orders or magnitude more than for sound, I'll admit, but computing
> power has increased by orders of magnitude in the last 20 years. So
> eventually, while it may still be technically possible to build a
> faster GPU, it won't make economic sense vs an integrated solution for
> most uses.
I guess that NVidia is trying to move in every field anyway
(integrated chips as well).
I personally haven't tried to really use CUDA or OpenCL ..so, I can't
tell what's the ratio between programmability and performance..
One thing that worries me a bit it's how NVidia is trying to cover
both the hardware and the software bases. By competing at every level,
it's not making too many new friends now..
The competition with chip makers is already intense, and coming out
with all these proprietary software solutions is going to make other
developers less interested in investing on NVidia's hardware and dev
tools.
ummmummm
Davide
Davide Pasca
> On Dec 19 2009, 9:27 am, Davide Pasca <dpa...@gmail.com> wrote:
>
> > At the end, one person asked about coping with cracks introduced in
> > the splitting stage..
>
> > And the answer was that they are currently not dealing with cracks
> > (this is also written in the paper). And the cracks are often hidden
> > by the high (13x13) sub-sampling.
>
> That's a fairly common misunderstanding of the problem :( Cracks (99%
> of them anyway) are not hidden by oversampling. They are merely
> antialiased better and are still very much visible in many cases. But
> that's to be expected if they haven't looked at the problem.
To be fair, I have to say the person that asked suggested about
generous sampling making up for the cracks. But I don't that that was
intended as an actual solution 8)
> > But the answer somehow ended up being on some performance figures,
> > basically saying that their compiler produces code that uses less
> > registers than Pixie.
>
> Weird answer indeed :) But the number of opcodes is not really
Could have been me reading the question or the answer wrong..
> relevant if you have enough resources to implement them as they are
> mostly independant. And I figure Pixie's compiler doesn't care that
> much about registers because it doesn't really have to.
That also sort of came out of the blue because there is no mention on
the paper or in the talk about Pixie.
Though I now see that Pixie is credited at http://renderants.org
> > GPUs. Network distributed rendering currently not supported and in
> > general not really trying to compete with the set of features of
> > PRMan.
>
> That would be quite crazy anyway given the amount of code they must
> have poured into that soup ;-)
Yeah.. but hard to guess with no code available. Can't have everything
8)
Davide