Audio and Pointcloud

[jackwo...@gmail.com]

Hey guys,
I know this has been discussed briefly in old posts. But I thought maybe there has been some updates.

I'm having issues syncing the audio of the Brekel almebic exports in Blender. Has any one managed to sync the audio with their captures?

All my capturing has been done so I'm looking for a solution in post.
Many thanks,
J

[Jedidiah Mitchell]

I shot some music videos with Pointcloud v2 years ago and found a solution but I set it up in-camera -- you might be able to use part of the technique in post but it will be harder. 

Since the Kinect has a variable framerate (my system fluctuated between 28~32fps) I used a phone with a timecode generator on it positioned somewhere in-shot to give me a fixed reference to real time in the RGB data recorded by Pointcloud. I then took that image sequence and mapped it to a time-time curve in Houdini (any 3D app should work though). I set keyframes on the first and last frames of the song to match the timecode I could see on the phone, then scrubbed through the track looking for places where the phone timecode didn't match the playbar in Houdini and added keys to nudge it back into place. With a few dozen keyframes I had a very steady match. I then copied that time curve to the alembic sequence and my pointcloud lined up well. 

Without the reference of the phone in your RGB data this would be much harder, but if there are musical instruments or dialog you can sync up visually with the song it's possible you could at least get close. 

[jackwo...@gmail.com]

wow that's a cool tip thanks. Sadly we already shot but, I will try lookout for key moments that make sounds to try and resync using these as markers. Did you have issues resyncing dropped frames?

[Jedidiah Mitchell]

No problems with dropped frames that I can remember, but I had a pretty high powered system running it. I have had issues when driving the Kinect with a NUC or laptop in the past, but those were always on tests where I didn't really care about consistency. There are several possible ways to interpolate through drops in Houdini but they are all pretty advanced procedural techniques -- I'm not sure if similar workflows would be possible in another tool. 

Good luck with the sync -- it should be possible to at least get close if you can get decent keyframes at the beginning & end to start with. Is it a narrative scene? That's definitely harder than music...

[Brekel]

Brekel PointCloud v3 internally uses a different way of synchronizing things (for both recording and working with files on the timeline), which should be much more precise.

I'm not 100% sure if all of that is used for older v2 files but they should at least work, worth a try to see if it produces better results for you I'd say.

Op donderdag 17 juni 2021 om 16:46:42 UTC+2 schreef jedmi...@gmail.com: