How to simultaneously scan with multiple sensors

[Brandon Pomeroy]

Hey all,

I've been lurking for a while, trying to figure out all the tricks with Skanect. A question I see on the forum quite a bit is "Can I use multiple sensors with Skanect?"

Unfortunately, it seems that the current answer is no. Hopefully that will change someday, but for now, you can use this nifty workaround!

Step 1: Open up multiple instances of Skanect

Step 2: Scan the same object from different angles simultaneously

Step 3: Load meshes into Meshlab, and align them

Step 4: Have a complete model!

Step 1 was the tricky part, but I've figured out a way around Skanect's "An instance of Skanect is already running" error using a program called "Sandboxie".

Sandboxie creates an environment for a programs to load into that are not allowed to communicate with the rest of your machine. You'll need the full version of Sandboxie to make this method work (Since you need multiple sandboxes...)

To get started, install Sandboxie, and create as many sandboxes as you need (I used 2, "DefaultBox" and "DefaultBox2".

1) Plug in your Kinect (or other sensor) to an available USB port

2) Right click Skanect, and click "Open sandboxed"

3) Select "DefaultBox"

4) Skanect should open and detect the sensor as normal.

5) A pop up window will appear, asking if you want to recover files. Click the checkbox that says something like "Do not ask to recover files until the end of the session", then close the window.

6) Plug in your second Kinect. Be sure that it is plugged into a USB port run by a different USB host controller, or you'll have bandwidth troubles.

7) Right click Skanect, and click "Open Sandboxed"

8) Select "DefaultBox2" (Or any other box that is not already running an instance of Skanect)

9) WAIT. 3 windows should pop up. The Skanect app, the recovery window (As mentioned before), and a small window that says "Failed to initialize sensor".

10) Close the recovery window as before

***This is where the process is likely to fail... the Skanect app should show a yellow sensor icon in the upper right hand corner. If, at any point, that sensor turns red, close the instance, unplug the sensor and start from step 6***

11) Bring the "Failed to initialize sensor" window to the front. Close it by CLICKING THE RED X IN THE TOP RIGHT CORNER. If you press "OK", the sensor will turn red, and you will have to start from step 6.

12) Wait. The sensor on Skanect should still be yellow, and you should get a "Starting Sensor" bar on the bottom. Wait for that to finish loading completely before clicking anything else

13) Scan simultaneously! 

14) The exported models can be recovered from Sandboxie after the instances are closed.

------------------------------

I've used this method pretty consistently, but it's not perfect. Sometimes, Skanect just won't see the sensor. Closing the sandbox and re-opening usually fixes that.

Aligning the scans is still an issue that we face... it's difficult to get it just right, and there's a lot of noise on the edges of the scans.

It's not a perfect method, but until there's native support for multiple sensors (Hint hint!), it's the next best thing.

Let me know if you give it a try, and what you think!

Thanks,

Brandon Pomeroy

[Nicolas Burrus]

Wow, this is definitely a great way to put more pressure on us doing
proper multiple sensor integration :-)

Thanks for sharing these tricks. We could definitely make your life
easier immediately by allowing multiple instances of Skanect running
at the same time with some special command line flag.

We'll give this some thought!
Nicolas

> --
> You received this message because you are subscribed to the Google Groups
> "Skanect" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to skanect+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Brandon Pomeroy]

I'm glad to hear that this puts some pressure on! Out of all the scanning software out there, I feel that Skanect really gets the best resolution, and has the smoothest workflow. We've been using a different system (that supports multi-sensor), and it's a bit... clunky in comparison. Unfortunately, our project requires multi-sensor, so we're stuck with it. If you ever need a beta tester for that multi-sensor integration, let me know! ;)

Command line flags would be awesome! Something like "/skanect.exe -u kinect01", where "-u" tells Skanect which sensor to use. I have no idea at all how difficult that would be to implement, but it would certainly reduce some headaches on my end!

Thanks,

Brandon Pomeroy

[Michael Balzer]

This is great Brandon.  I too have found other software products clunky and ReconstructMe is just getting around to adding the features included in Skanect and the only way you can support multiple scan heads is to build your own app with their SDK.

I would second the ability to use a command line feature as well, and I like Brandon's idea with the command line flags.

[Brandon Pomeroy]

Time for part two of this "tutorial"! I just confirmed this works today, so I'm excited to share.

I lamented in the first post about the difficulties in aligning the separate meshes. I kept wanting to just throw the meshes into Skanect, and have it do it's nice smoothing algorithm on them! This, of course, isn't possible.

I was digging around in the way Skanect saves it's files. It's basically a folder FILLED TO BURSTING with data from each frame that was taken during your scan. There didn't seem to be any rhyme or reason to the folders, other than them being named in a chronological order.

I ended up grabbing some images from two random scans, and pasted the second scan images into the first scan source. I expected Skanect to complain, but, happily, this wasn't so! The scan loaded, and trying to do offline reconstruction resulted in exactly what one might expect: It went through all the images of the first scan, then moved on to the images in the second scan. The scans had, luckily, been fairly similar, so tracking wasn't lost. The resulting mesh was... horrifying, since it was basically two people mashed together.

The question was now "How can I use this to aid in multiple sensors?" Obviously, each sensor can grab a scan at the same time, and the images can be thrown together to make a single mesh!

To test this, I set up just one Kinect, and scanned a person from 3 different angles (120 degrees between - Front, back right, back left). The scan began at the subject's middle, went up, then down, then the subject was rotated to the next position, then the scan went back up to the subject's middle. This was done so that the end of the first scan was in the same location as the beginning of the next scan.

The end result: I had a 1.5 GB folder of data that Skanect slowly reconstructed. There were some issues with my subject not standing perfectly still, but it was much better than aligning by hand! I imagine that if I run 3 scans simultaneously, the alignment would be nearly perfect.

We may not be able to combine meshes in Skanect, but this is the next best thing!

[James Allan]

Sounds great Brandon. I wonder if it would end up being quicker to take the two or three meshes from Skanect and align them in Meshlab.

[Brandon Pomeroy]

I haven't had much luck getting the edges of scans to be good enough to blend... I always get weird artifacts. The other advantage to this method is that it's completely automatic. (Unless meshlab has a secret "auto-align" button that I've missed).

The end goal for me is to push one button, and have a computer do the rest of the work. If it takes an extra five minutes of a loading bar, but I don't have to think, I'll take it! ;)

[Brandon Pomeroy]

Who wants to read a bunch of words? Not me!

I put together a little video outlining my technique. This is using 1 sensor, scanning 3 times, but it's a small jump to turn this into using 3 sensors each scanning 1 time.

Here's a link to the video. The audio's not great, but it works.

Here's a link to the part exported to Sketchfab. Not too terrible!

So far I have only had the capabilities to run sensors in series. I hope to test this running the sensors in parallel, using the sandboxie method mentioned above. I just have to get my hands on a computer with 3 USB host controllers! I expect some complications in the file names if they are captured at the same time, but hopefully it won't matter.

Thanks,

Brandon Pomeroy

On Sunday, November 3, 2013 10:25:52 PM UTC-8, Brandon Pomeroy wrote:

[James Allan]

Again, excellent. Thanks for sharing Brandon. I'm dreaming of a multi-sensor setup now :D

[Nicolas Tisserand]

Yes, thanks Brandon, feature pressure received! :-)

[eyebee]

Nice work Brandon! Just shows where there's a will there's a way.

Really appreciate the share.

[Michael Balzer]

I have been doing the same thing with textures. Right now I am limited to down sampling the hi-res texture images to 640 X 480, but FUSION will use these just as effectively.  The advantage until Nic makes an SDK or switches in the software, is less noise, better color and less banding.

[Mark Rogal]

Two years later and progress on multi sensor use ?

[Count Montecristo]

The year is 2016.  

Skynet and the Terminators have been defeated.

Marty McFly's kids never got in trouble due to the hoverboard landing a bully into jail.

Another generation of Kinect (V2) is available for the Xbox ONE...

      ...Yet, I cannot seem to find anything on Skanect offering multiple cameras in response to supposed pressure which they shared was present for implementation.

Soooo... What do you say guys, the week before global warming erradicates us as a species?

[Manuel Guillermo Fraga Castro]

I agree with you
We bought the program last year and have been waiting for this functionality because it would solve us many problems. However we are likely to leave Skanect and look for other software

[Primera]

So,,,,is multiple sensors available yet?

[scott...@occipital.com]

Skanect (still) supports a single depth sensor and, if multiple sensors are connected, it will just pick the first one it finds; there is no option to select a specific sensor either.

Multiple simultaneous sensor support is a common feature request, so you are not alone in requesting it, but I am not aware of this feature being approved for any near-future releases of Skanect.

-Scott

[Primera]

That is what I assumed

But as I posted in my other thread...mine doesn't not select one, but hops between the 2

[Tobby Ryan]

Primera, 

That is correct. It DOES hop back and forth during the live scan. This makes the live pretty much unusable for the live output.. 2 things can be done to fix this in a sense.

1. Offline Reconstruction on either of the internal folders.One at a time

2.You can run multiple instances of skanect, one with each camera and scan with each camera on the same object.  On Windows the following command line switch is required: "--allow-multiple-instances"  

This of course will allow each of the scans in seperate folders instead of seperate folders with the same SKN folder. (add the switch after the skanect.exe" )

You can then take the 2 outputs and MESH them together manually in another program.

This may not be exactly ideal, but hey, its also doable.

Of course when using 2 cameras framerate will drop some. :)

Please forgive me Scott for mentioning this.

[Tobby Ryan]

[Tobby Ryan]

I forgot to add.... Only hook up 1 scanner during opening the first instance, Open up the second instance, THEN connect that scanner. This way it will lock each scanner into each instance.