[eq-dev] Updating various data to clients.

[Petros.Kataras]

Hi,

I was wondering if there is a recommended way to take care of updating
various heterogeneous data .

i.e data could be timestamps of videos for syncing across multiple nodes or
attributes of a particle system and so on ..

Should all these stuff be handled from ViewData::update() in sequel terms ??

Thanks for any hints..

Best,

Petros



--
View this message in context: http://software.1713.n2.nabble.com/Updating-various-data-to-clients-tp7585908.html
Sent from the Equalizer - Parallel Rendering mailing list archive at Nabble.com.

_______________________________________________
eq-dev mailing list
eq-...@equalizergraphics.com
http://www.equalizergraphics.com/cgi-bin/mailman/listinfo/eq-dev
http://www.equalizergraphics.com

[Stefan Eilemann]

Hi Petros,

On 7. Jul 2014, at 15:37, Petros.Kataras [via Software] <ml-node+s171...@n2.nabble.com> wrote:

> i.e data could be timestamps of videos for syncing across multiple nodes or attributes of a particle system and so on ..
>
> Should all these stuff be handled from ViewData::update() in sequel terms ??

Very likely yes, since this gives you consistency wrt the rendered frame. You could update your objects using Collage functionality only, but this would run asynchronous to the rendering. For global state there is also the FrameData. In general people tend either to use a central registry (aka co::ObjectMap) or a tree-like structure of objects attached to FrameData or View.

I'm not sure if that answers your question.


Cheers,

Stefan.
--
http://www.eyescale.ch
https://github.com/Eyescale/
http://www.linkedin.com/in/eilemann






signature.asc (858 bytes) <http://software.1713.n2.nabble.com/attachment/7585915/0/signature.asc>




--
View this message in context: http://software.1713.n2.nabble.com/Updating-various-data-to-clients-tp7585908p7585915.html

[Petros Kataras]

Hi Stefan,

once again thanks for your prompt reply and for bearing with me !

Maybe a bit of more info would help illustrate what the target is..

We have a project that will consist of 27 displays connected to 9 -
renderers + 1-master each renderer handling 3 - screens with an NVIDIA
Quadro K5000.

The project will be heavily media-content based i.e videos, images,
animations, effects etc.

Although this is not a usual EQ test case we have decided to go with it
since we still believe that this would be the best option for various
reasons.

As a result this means that there is gonna be heavy development the next
2 - 3 months with SEQUEL as a base in order to create an interface that
works relatively intuitively for people that havent worked before with
EQ since there are more than 3 engineers that are gonna be working on
this installation and have no experience at all with EQ.

Back to the main topic now.. Ideally I would like to have an app
interface that is as close as possible to a usual interactive single
threaded opengl application ( i.e setup, update, render, handle events )
since this feels more familiar to someone who hasnt worked before with EQ .

For setup and draw things are quite clear I believe but for updating I
would personally prefer to expose an update function from inside
seq::App . I was looking at seq::masterConfig::run and was thinking if
it would be an option to update the various data between frameFinish and
frameStart ..

Would that be a valid option you believe ??

Thanks,

Petros

On 2014-07-08 10:41, Stefan Eilemann wrote:
> Hi Petros,
>
> On 7. Jul 2014, at 15:37, Petros.Kataras [via Software] <ml-node+s171...@n2.nabble.com> wrote:
>
>> i.e data could be timestamps of videos for syncing across multiple nodes or attributes of a particle system and so on ..
>>
>> Should all these stuff be handled from ViewData::update() in sequel terms ??
> Very likely yes, since this gives you consistency wrt the rendered frame. You could update your objects using Collage functionality only, but this would run asynchronous to the rendering. For global state there is also the FrameData. In general people tend either to use a central registry (aka co::ObjectMap) or a tree-like structure of objects attached to FrameData or View.
>
> I'm not sure if that answers your question.
>
>
> Cheers,
>
> Stefan.

[Stefan Eilemann]

On 8. Jul 2014, at 11:42, Petros Kataras <petros....@aec.at> wrote:

> We have a project that will consist of 27 displays connected to 9 - renderers + 1-master each renderer handling 3 - screens with an NVIDIA Quadro K5000.
>
> The project will be heavily media-content based i.e videos, images, animations, effects etc.
>
> Although this is not a usual EQ test case we have decided to go with it since we still believe that this would be the best option for various reasons.
>
> As a result this means that there is gonna be heavy development the next 2 - 3 months with SEQUEL as a base in order to create an interface that works relatively intuitively for people that havent worked before with EQ since there are more than 3 engineers that are gonna be working on this installation and have no experience at all with EQ.
>
> Back to the main topic now.. Ideally I would like to have an app interface that is as close as possible to a usual interactive single threaded opengl application ( i.e setup, update, render, handle events ) since this feels more familiar to someone who hasnt worked before with EQ .
>
> For setup and draw things are quite clear I believe but for updating I would personally prefer to expose an update function from inside seq::App . I was looking at seq::masterConfig::run and was thinking if it would be an option to update the various data between frameFinish and frameStart ..
>
> Would that be a valid option you believe ??

What happened to attaching the data to the ObjectMap? As far as I can see, I would just manage the objects as we discussed in https://github.com/Eyescale/Equalizer/issues/277 through the application or renderer - or am I missing some complication detail?

For videos you likely want to load them from a shared filesystem and sync only the timestamp.

What is the state of #277? From a quick luck it seems this one is in good shape. Can you open a pull request if you think it's ready to merge?

[Petros.Kataras]

Hi Stefan,

The data will be attached and handled by the ObjectMap . The question now is
what would be the best way to update aka commit / sync these data on a per
frame basis since there are data that would have to be updated on a per
frame basis ( video timestamps for example ).

That's why I am thinking of exposing an update function through
seq::app::MasterConfig between frameFinish and frameStart so that the user
can set a dirty state of the data on a per-frame basis.

Basically this is already happening in ViewData::update but I would prefer
to expose a similar mechanism through seq::Application instead.

As for #227 I am not entirely sure if it is 100% complete.. It does expose
the full internal ObjectMap interface for the application and renderer i.e
register/deregister, map/unmap but this
<https://github.com/Eyescale/Equalizer/issues/277#issuecomment-42765444>
is still unclear to me..

Obviously I have failed to describe properly what my question / problem is
in that case since your answer
<https://github.com/Eyescale/Equalizer/issues/277#issuecomment-42768120>
is obvious to me but it doesnt explain my problem :)

This is also why I asked for an illustration if you had some time of
registering / mapping an arbitrary number of objects based on the example
that I have created there.. :)

Anyways, I am gonna get back to this with some fresh eyes and hopefully its
gonna be clear..

Obviously if you think that what is there already is enough for you I can
open a pull request any time..

Thanks,

Petros.



--
View this message in context: http://software.1713.n2.nabble.com/Updating-various-data-to-clients-tp7585908p7585953.html

Sent from the Equalizer - Parallel Rendering mailing list archive at Nabble.com.

[Stefan Eilemann]

Hi Petros,

On 10. Jul 2014, at 11:12, "Petros.Kataras [via Software]" <ml-node+s171...@n2.nabble.com> wrote:

> The data will be attached and handled by the ObjectMap . The question now is what would be the best way to update aka commit / sync these data on a per frame basis since there are data that would have to be updated on a per frame basis ( video timestamps for example ).
>
> That's why I am thinking of exposing an update function through seq::app::MasterConfig between frameFinish and frameStart so that the user can set a dirty state of the data on a per-frame basis.
>
> Basically this is already happening in ViewData::update but I would prefer to expose a similar mechanism through seq::Application instead.

Ok, that makes sense now. I guess you wouldn't set a dirty state but rather do processing which happens to make some objects dirty.

> As for #227 I am not entirely sure if it is 100% complete.. It does expose the full internal ObjectMap interface for the application and renderer i.e register/deregister, map/unmap but this is still unclear to me..

> Obviously if you think that what is there already is enough for you I can open a pull request any time..

I would guess that you have a use case against which you validate the code. I'll review it carefully, but won't have the time to create a test or example.


Cheers,

Stefan.



signature.asc (858 bytes) <http://software.1713.n2.nabble.com/attachment/7585954/0/signature.asc>




--
View this message in context: http://software.1713.n2.nabble.com/Updating-various-data-to-clients-tp7585908p7585954.html

[Petros.Kataras]

> Ok, that makes sense now. I guess you wouldn't set a dirty state but rather
do processing which happens to make some objects dirty.

Exactly..

> I would guess that you have a use case against which you validate the
> code. I'll review it carefully, but won't have the time to create a test
> or example.

Right now there is minimal example ( inside seqPly ) illustrating
registration and committing of master instances UUID's to the slave
ObjectMap. Mapping appropriately is where I am losing track. Would you like
to review what is already there i.e should I already open a pull request for
you to review or would you prefer a more concrete use case ??

Cheers,

Petros






--
View this message in context: http://software.1713.n2.nabble.com/Updating-various-data-to-clients-tp7585908p7585956.html

[Petros.Kataras]

Hello from me again..

Development is on going on this project and I have a question which I would
like to bring here..

I am trying to find if there is a way to know on which context a deserialize
function of a shared object is called. What I mean by this is if there is a
way to know ( and be able to grab ) for example the specific slave (
renderer lets say ) inside the context of which the deserialize function is
being called at any particular moment.

From the look that I had I couldn't really find a way of getting this
information and I am hope that I missed something..

I would need this for mapping slave instances of custom shared object
classes inside deserialize otherwise I would have to somehow pass this info
around always and I would like to avoid this if there is a way to grab this
info properly..

Thanks for any insights !

Petros



--
View this message in context: http://software.1713.n2.nabble.com/Updating-various-data-to-clients-tp7585908p7586322.html

[Stefan Eilemann]

On 19. Sep 2014, at 17:11, Petros.Kataras [via Software] <ml-node+s171...@n2.nabble.com> wrote:

> I am trying to find if there is a way to know on which context a deserialize function of a shared object is called. What I mean by this is if there is a way to know ( and be able to grab ) for example the specific slave ( renderer lets say ) inside the context of which the deserialize function is being called at any particular moment.

Not sure what information you’re looking for. Does DataIStream::getLocalNode help? This can be cast into an eq::Client within an Eq app.

Cheers,

Stefan.

signature.asc (858 bytes) <http://software.1713.n2.nabble.com/attachment/7586328/0/signature.asc>

--
View this message in context: http://software.1713.n2.nabble.com/Updating-various-data-to-clients-tp7585908p7586328.html

[Petros.Kataras]

Hi Stefan and thanks for your reply..

Maybe a bit more info about this would help clarify what I mean exactly..

As discussed in previous sessions we are working with sequel as a base and
we have implemented a "LazyPointer" template to manage the mapping of slave
instances. The master UUIDs are serialized on the slaves and then for each
master instance a "LazyPointer" is created which holds the UUID of this
master instance of the object.

The mapping then occurs through operator overloading when the "LazyPointer"
is being accessed.. Something along these lines:

(snippet)

T* operator->()
{
if (!mPointer && mID != 0)
{
if (isMaster())
{
mPointer =
static_cast<T*>(globalState::getAppPtrBase()->mapObject(mID));
}
mPointer =
static_cast<T*>(globalState::getRendererBase()->mapObject(mID));
}
return mPointer;
}

where mID is the master instance UUID of the shared object. Now, the problem
is that the globalState interface that I have right know is implemented as a
singleton which obviously breaks in multi-pipe configs since in these cases
we have more than one renderer. On the other hand I would really like to
keep a global instance of the interface since this would make life much
easier for other people that have to use the interface and have no
experience with equalizer.

Do you have any suggestions on how it would be best to approach this case ??

Btw all these are on the context of a framework that combines equalizer with
openframeworks and has to be as simple as possible for other people to work
with, without any prior experience on the internals of equalizer.

Thanks again for your help,

Petros




--
View this message in context: http://software.1713.n2.nabble.com/Updating-various-data-to-clients-tp7585908p7586345.html

[Stefan Eilemann]

Hi Petros,

On 22. Sep 2014, at 17:33, Petros.Kataras [via Software] <ml-node+s171...@n2.nabble.com> wrote:

> where mID is the master instance UUID of the shared object. Now, the problem is that the globalState interface that I have right know is implemented as a singleton which obviously breaks in multi-pipe configs since in these cases we have more than one renderer. On the other hand I would really like to keep a global instance of the interface since this would make life much easier for other people that have to use the interface and have no experience with equalizer.
>
> Do you have any suggestions on how it would be best to approach this case ??

I would try to get the globalState from the eq::Pipe/seq::Renderer instead of a singleton. If that’s too cumbersome, you can use thread local storage to have a per-thread singleton. This assumes you want to maintain the objects per GPU render thread.

HTH,

Stefan.
--
http://www.eyescale.ch
https://github.com/Eyescale/
http://www.linkedin.com/in/eilemann

signature.asc (858 bytes) <http://software.1713.n2.nabble.com/attachment/7586347/0/signature.asc>

--
View this message in context: http://software.1713.n2.nabble.com/Updating-various-data-to-clients-tp7585908p7586347.html

[Petros.Kataras]

Hi Stefan,

Thanks for the suggestions!

I decided to go the thread_local way which seems to work fine so far and I
think will save me some time to focus on other aspects of the framework ..

Thanks again..

Petros



--
View this message in context: http://software.1713.n2.nabble.com/Updating-various-data-to-clients-tp7585908p7586352.html