Fuse PCv3 data with BBv3 possible?

[Frank Spalteholz]

Dear Jasper, we are currently developing an VR application where we are using PCv3 for recording point-cloud data of a human doin some performances. SInce we are processing those clouds within Houidni, it would be super handy having at least some informations about the velocities. We are currently "faking" this by merging 2D-velocity (for x,y) from the RGB-video + projecting a plane onto the obj-sequence for getting z-velocities (that's "kind of ok-ish). So we are thinking about to fuse data coming from Brekel Body (using bones-animation ->  applying at least a "thresholded velocity-area" around the bones to manipulate the point-cloud) But to make this work (and please correct me if I'm wrong) I'd need to load a recorded PCv3 take in BBv3 for processing. But is this even possible? I was installing the BBv3 Trail but i couldn't find any "Timeline Tab" for testing this myself. Thanks again!

[Brekel]

Hi,

Sorry but there is no velocity data available in the pointcloud data.

Brekel Body v3 does not have a timeline but you can record body data & pointcloud data (BPC which can then be loaded/converted with PointCloud v3) simultaneously in BB3 if that helps.

Op woensdag 6 juli 2022 om 09:00:40 UTC+2 schreef piratfr...@gmail.com:

[Frank Spalteholz]

Missunderstanding here. We can calculate velocity from the bones later since they have consistent transforms over time where the pointcloud does not. So we are planning to skin a proxy-geo to the animated joints/bones and using this mesh to calculate velocities we can apply to the voxels from the point cloud that are within a certain treshold/distance to the mesh. Does this make sense?

[Brekel]

Ah I see, I misunderstood indeed.

If that's something you can do in Houdini that sounds plausible indeed.

Op woensdag 6 juli 2022 om 13:29:05 UTC+2 schreef piratfr...@gmail.com:

[Frank Spalteholz]

Awesome! " Brekel Body v3 does not have a timeline but you can record body data & pointcloud data (BPC which can then be loaded/converted with PointCloud v3) simultaneously in BB3 if that helps." <- That's excellent news Jasper! And I've found all needed settings now. I assume using 5 Azure Kinect (as long as we made sure using proper hardware of course) is also no problem for BBv3? If so we will get us a license.  

On Wednesday, July 6, 2022 at 10:43:19 AM UTC+2 Brekel wrote:

[Brekel]

For multiple Azure Kinects you will need a machine that can handle the USB bandwidth, generally speaking you can run one or two sensors at most on the motherboard's USB ports of a desktop machine. A StarTech PEXUSB3S44V (or similar) card will add 4 additional full speed ports.

For Body tracking with multiple you will also need a beefy NVIDIA graphics card as this is quite heavy to process.

Op woensdag 6 juli 2022 om 13:43:08 UTC+2 schreef piratfr...@gmail.com:

[Frank Spalteholz]

I see. So what about the following setup: We are going to use 5 alienware r13 pc's (with intel 9 16corfe, 64gb ram, nvidia 3090rtx each) each of this machine sould have proper hardware specs (i guess we could attach 2 azure per pc but i'm not sure what the usb-controller will be able to handle so in worst case we can use 1 pac per cam as i said. So to get 5 point-clouds recorded + a proper skeleton anim what about using 1-2 machines with BBv3 and the others "just" using BPCv3? I guess we will have to synch the records later by hand between the BBv3 and BPCv3 streams but that's no problem. What you think?

[Brekel]

You can use multiple machines as well, in practice fewer (beefier) machines are easier than more (slower) machines.

Op woensdag 6 juli 2022 om 14:05:53 UTC+2 schreef piratfr...@gmail.com: