Brekel PointCloud 3 BETA 0.84 64bit - Realsense D435i - Calibration and Settings bug

[Frank Spalteholz]

Dear Jasper,

first of all ... amazing tool! But i have some starter issues you hopefully will be able to help me with:

1. My setup is using a Razer Blade laptop with i7 2.20GHz, 32Gig Ram,  GeForce GTX 1060

(and this inbuild Intel whats so ever)

2. I'm using 2 Realsense D435i (wich are running the latest firmware + installed Realsense SDK 2.49)

3. I can confirm that PC3 can detect both cams without any problems but i can't change the sensor-settings(live) neither depth nor video without having the program hardly crashes. 

4. The default settings for depth and video are (848:480 30fps -> depth) and 1280x720 30fps video. You can see in the 3D view that (maybe) because of teh missmatch of those 2 resolutions the video isn't correctly mapped onto the cloud. I was having the same issue with PC2 but when switching the video-res to match depth life was good. 

I'm writing this because calibration unfortunatly doesn't work either which i assume might be due to those 2 missmatching depth and video input streams. But maybe you are not fusing those 2 streams and using "only" rgb but then i'm lost why the calibration pattern doesn't get recognized. I was testing both cams together in multi-sensor mode with static and moving marker mode, also i've tested each cam individually + each on different usb-ports + with 5 different cables (just to double check the whole setup) but with no luck. My cams are are more or less 30cm apart in a kind of stereo-vision like setup (eyeball-paralled because i'm asuming that this doesn't matter) The calibration pattern is printed on A4 with more or less precise 177mm and placed about 50cm away from both cams. 

Thanks so much for your help in advance!

Frank 

[Frank Spalteholz]

Ok ... i was testing once more and recorded 2 sessions (each cam, 848x480 60fps depth and video) with PC2. I was then importing them to PC3 + matching timeline (which is a great option btw). I can then confirm to get an offline-calibration to work. It has some obvious offsets but that doesn't matter for now. So it seams that fitting the resolutions could be the key here but then it would be even more important to get the setting-changes on the cams to work. So again, i'd really appreciate your help on that. 

Thanks again!

[Frank Spalteholz]

Hi Jasper,

to make sure that this is a general issue and not just related to my specific laptop i was testing everything on my workstation again (AMD Ryzen 9 5950X 16-Core Processor 3.40 GHz, 64,0 GB Ram, NVIDIA GeForce RTX 3080, Driver 466.77) -> same issue. I can confirm to successfully connect both cams -> live streams are visible but when i try to change the cam settings either depth or video i got an instant crash

[Brekel]

Just tested here with a 435 (don't have the i version) and I can also make it crash, will put it on the bug list to be fixed.

Note that those default resolutions are selected based on the native resolution for the Intel sensor types, you can bump 'm up (well if it didn't crash) but that generally doesn't produce more actual detail.

Op zondag 3 oktober 2021 om 14:13:53 UTC+2 schreef piratfr...@gmail.com:

[Frank Spalteholz]

Thanks for checking that for me so fast! It would be good if i could save out custom settings via Realsense-Viewer as json and load them inside PC3 but that's maybe too much to ask for since those D400-series-cams are no longer produced so i could understand that those types aren't the once you will prefer in the future (I still find them usefull and Azure Kinect still aren't available). So is there any way to force PC3 connecting those sensors with other default values so i won't have to use the setting-options? 

[Frank Spalteholz]

Hey Jasper. I'm planning to use PC3 in a production early In Nove (around 11th) and without any pushing I'm wondering if this setting-change issue can be solved within the next itteration/version? Also when exactly will it be available for testing? Thanks so much again! 

Best wishes

Frank

[Brekel]

Unfortunately I have not been able to find a fix for this, but I'll keep trying.

Op maandag 25 oktober 2021 om 11:17:57 UTC+2 schreef piratfr...@gmail.com:

[Frank Spalteholz]

Thanks Jasper! I've ordered me a L515 in the meantime also hoping to achieve better quality. I saw that this model is also 

supported by PC3. Can you confirm that this bug is not present using the L515? 

[Brekel]

The problem lies with the RealSense library returning a null video stream after I set the resolution and request if it actually did it so it crashes for all RealSense sensors.

The app does pick the best default resolution for each RealSense sensor, based on the native/optimal color/depth sensor resolutions. The drivers/SDK can deliver upscaled results but they will not yield better quality (while being slower).

Op maandag 25 oktober 2021 om 20:29:08 UTC+2 schreef piratfr...@gmail.com:

[Frank Spalteholz]

Understand so let me again explain the bigger problem here and maybe we can find a workaround in the meantime cos i really would like to use your tool for our upcomming production-shoot. My initial problem was that i couldn't get my 2 D435i calibrated anyways which calibration-method i was using (online/offline) So i was testing a bit and since the rgb resolution and the depthcam-resolution are different i've tested to record each cam seperately with PC2 and imported those 2 streams into PC3 (both cams were running with 848:480 30fps for depth and rgb) So after import i could use at least the offline calibration method in PC3. So now i'm a bit lost. With PC3-default-settings i can record with my 2 D435 in synch but without calibration but maybe i've missed something or made a mistake on the calibration-part. 

[Frank Spalteholz]

Hi Casper,

tried the new version with 2 L515 in the first place but i can't record cos its hardly crashing the moment i press "record" (left all brekel-settings to default). Tried also with just one L515 and then with just one D435i but it's the same here. With tha last version i still had the callibration issue but could at least make a record which is a bit unfortunate now. 

[Frank Spalteholz]

Sorry Jasper for misspelling your Name last time. Don't mind pls but i'm wondering if you've found the time to check if any Realsense device is working with your last version? 

Thanks

Frank

[Brekel]

Thanks for pointing out the recording crash, this should have been solved by the latest version now.

Op woensdag 10 november 2021 om 07:54:00 UTC+1 schreef piratfr...@gmail.com:

[Frank Spalteholz]

Hi Jasper, first of all i can confirm that i'm now able to successfully capture pointclouds using 4 realsense devices in parallel without any crashes so far which is GREAT! Inbuild calibration still doesn't work but i could calculate the extrinsics myself and exporting a new Sensor_Alignment.txt. By checking the format of your calibration file I've found a "transform matrix" (in homogenious form) and i assume i will have to provide the rotation and translation data there relative to the marker. The only thing till now that i'm wondering of is that this matrix has 17 entries instead of 16 so what is the first entry used for? Is this an index for a matrix-sequence in case you will have several cameras? Thanks a lot for your great work! Best wishes Frank

[Frank Spalteholz]

Oh never mind (i was distracte by the formating of the file). the first entry is the tilt value right? But why is this needed / what is this tilt-value used for and how do you calculate that?

Best wishes Frank

[Brekel]

Hi Frank,

The alignment file format is not officially documented as it may be subject to change.
However, as you already found out, they are simple text files and there is a description in the header on what the data represents so you can hack around with them.

The tilt value is only used for Kinect v1 (XBox 360) sensors and it describes the internal motor position that tilts the sensor forward/backward, for other sensor brands/types this value is ignored so you can simply leave it at 0.

Hope that helps,

Op donderdag 13 januari 2022 om 10:32:44 UTC+1 schreef piratfr...@gmail.com: