Kinect for Windows v2 support

[Nicolas Burrus]

Dear Skanect users,

As promised last week, we’ve been spending more time investigating Kinect for Windows v2 to give a clear answer on when we will support it.

We had already spent some time trying to integrate it into Skanect a few months ago, and back then the scanning results were not good enough for us to support it.

So we’ve been waiting on Microsoft to release an update to its SDK to see how things are improving. Version 2.0 was published a few weeks ago, so now was a good time to come back to it and try again. And unfortunately, it is not yet where it needs to be to justify a full integration in Skanect.

Kinect v2 uses a different sensing technology from Kinect v1 (time-of-flight instead of structured light), which has a number of advantages, including an improved spatial resolution, which is ideal for better gesture recognition / skeleton tracking. And since this is the primary target of the Kinect team, v2 is definitely a big improvement over v1 in that respect. However for 3D scanning, it also means more sensitivity to the material and color of the objects, and more noise. And it turns out that with the current SDK, it is still hard to get good quality scans of objects and people.

Here are two of the experiments we have made using the Microsoft Kinect Fusion samples to show how it performs (last time we were using Skanect engine, but this time we wanted to make sure Skanect is not the issue).

The first experiment is a frontal bust scan (not doing a full 360 to avoid adding tracking issues). While there are arguably more details than with v1, it tends to create an “alien”-like look, and has a hard time with eyes and hair.



​

The second experiment is looking at a book, which is perfectly flat, but has some text with different colors on the front page. Here you can see clearly the black “Foot” word on it, which appears to be a different depth as the rest.


​

We will periodically revisit to see if it makes sense to change our position, but taking these experiments into account we have decided to not support Kinect v2 for now, and focus our energy on areas where we think we are more likely to improve the overall 3D scanning experience.

We understand the frustration for those of you who already bought Kinect v2 sensors, but we hope you will understand our position. As usual we’ll welcome your feedback, especially if your experience with Kinect v2 has been very different so far and if you found a way to do great scans with it.

The Skanect Team

[Lars Carlsson]

Thank you for this post! Even if it is disappointing I understand it more than well.

 All the tests I have performed with Fusion and 3D builder points the same way as yours.

So hopefully MS will update the SDK and make the hardware more usable for scanning.
Right now it is just an expensive but worthless piece of junk sitting on top of my monitor...

Thank you for your efforts anyway!! I am still using my old scanner with Skanect and are getting great results.

Lars

[b2unit2]

Nicolas,

Thnx for the expenation.
And thnx fot the work you put into this .

I was thinking, almost every with an kinect 2 owns a kinect 1
What if you connect both , kinect 2 and kinect 1 and use the information from the kinect 1 to counter the color problem. You could just "stack" the sensors...
The 2 diffrent techniques wouldnt bite earch other rigt?

I know you do not support mutiipel sensors but maby they wont intervene with eatch other.

I stil not losing hope!
Ceep up the good work!

Greetings,

Rob

[Richard McMichael]

Would it be possible for yall to create a beta for current Skanect users for the v2 implementation? (I'm not a current Skanect user but I'm still interested in making the purchase as I have the v1 and v2 ((I've been a free or open source software kinda guy - but feel an investment in yalls product is also worth it))). 

Also have yall talked with photographers on techniques that they use to polarize and depolarize things with color (which in my experience removes some affects) It may help to fix problems like with the book.  

I ask these things because I've been doing research for a while on 3D modeling and I ended up going out and buying an Xbox One just for the Kinect because of its potential, yalls program, and the 3D builder app within Microsofts App store... all of which have tremendous opportunity for development of real world stuff.  I have no doubt that yall will get a fix for it, yall just dont have the resources to do so...  Could you create a beta app and let your users experiment with techniques to improve the software in exchange for portfolio work and also allow a base of knowledge to be formed to fix these problems?  

[Marcus Wolschon]

Damn,

the objects I need to scan in the highest resolution are all single-colored and non-reflective (in the IR range) anyway. :(
Technical parts with curves that I cannot easily remodel using just a caliper and some time.

[Nicolas Burrus]

Hi Richard,

We are not planning to ship binaries since it would require extra work for the packaging. However you can simply try the Kinect Fusion samples that come with the Microsoft SDK. It will give you a good idea of how scanning with Kinect for Windows v2 would look like at this stage.

--
You received this message because you are subscribed to the Google Groups "Skanect" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skanect+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nicolas Burrus]

Hi Rob,

The two modalities (Structured light in v1, Time-of-flight in v2) are
for sure complementary, and merging them could potentially give better
results. However this seems more like a research topic at this stage,
and would not be a very convenient product to use.

Anyway, we strongly think that we can bring scanning to the next level
anyway, even with the current Structured light technology.

[TechLite]

Thanks very much for letting us know Nicholas. 

I was hoping that the Kinect V2 would be included, but my experiments with the Kinect Fusion program in the SDK had similar results to yours. I decided to return it to MS because of the lack of clarity from the HD camera and the poor tracking with their Fusion software. I'm now a Structure Sensor/iPad owner and it tracks very well and scans were easy.  I'm happy I made the switch especially given your K2 roadmap decision.

That being said, I don't understand why a little free program like ITSEEZ3D has remarkably better textures than Skanect Pro. I hope you look into that next rather than spending any more time on the MS Kinect V2.

Cheers,
Bob

On Friday, November 21, 2014 2:16:56 PM UTC-5, Nicolas Burrus wrote:

[Nicolas Burrus]

Hi Bob,

Thanks for sharing your feedback. We are working on leveraging better the very good color camera of the iPad. And indeed, we also think it's a better use of our time at this stage.

--

[TechLite]

Good to hear Nicolas. Better color textures will be very welcome indeed. Looking forward to it.
For now I'll use ITSEEZ3D for fine detailed head shots and Skanect for larger, full body shots. I don't like having to use ITSEEZ3D's online rendering service and prefer the Skanect processing on my computer. Better security that way.
Bob

[MrRoxysc]

Dear Scanect Team

I understand Your decission but I want to suggest You that Microsoft made a serious competitor of Scanect.

Microsoft created a free application called 3D Builder which works even with kinect xbox 360.

Below is a link to the answer to the question of one of the customers by employee of Microsoft  if

3D Builder works with Xbox 360 Kinect:

http://answers.microsoft.com/en-us/windows/forum/windows8_1-winapps/is-3d-builder-work-with-kinect-360/3638b3df-26d3-4581-aee7-f6cfcd5175c4

Today I learned to scan with Kinect Snesor 2.0 with  3D Builder and can say that

the quality is great. I will try in the near future to send scans here.

I think you should reconsider Your decision

because you have a serious competitor and deposition of 

works on the compatibilisation of the new Kinect Sensor can reduce sale of Scanect  :(

I personally regret because  GUI of Scanect is very cool and user friendly.

In my small country people say, "who is in a place that goes backwards ...."

Best whishes 

Marek

Poland EU

[MrRoxysc]

P. S.

Please see this:

https://www.youtube.com/watch?v=nXbDkJ7cRrA

W dniu piątek, 21 listopada 2014 20:16:56 UTC+1 użytkownik Nicolas Burrus napisał:

[MrRoxysc]

Please see this film:

https://www.youtube.com/watch?v=nXbDkJ7cRrA 

W dniu piątek, 21 listopada 2014 20:16:56 UTC+1 użytkownik Nicolas Burrus napisał:

[Tobby Ryan]

I am getting great quality now that I have updated my video card.

Yes I went out and purchased an Xbox One Kinect for testing purposes with the USB Adapter as well. I will be happy to do any beta testing needed. :)

[Marcus Wolschon]

What software are you using with the Kinect V2 for XBox?

What results do you get with surfaces that reflect IR light differently? (see the book cover that Skenect posted)

--
You received this message because you are subscribed to a topic in the Google Groups "Skanect" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/skanect/j8-hqrZIu-U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to skanect+u...@googlegroups.com.

[Tobby Ryan]

the only 3 software packages i have seen sofar for the Kinect v2 is:

1.  The Kinect 2.0 SDK
2.  3d builder (for win8.1 from the microsoft store) its okay, but requires internet access for repairs for some reasson)
3.  kscan3d, which just takes several captures and you select to stitch together. (nothing automatic)
out of the 3, the 3d builder is the better. but i cannot figure out how to export colorized models from it.

[Marcus Wolschon]

What are your results with 3D Builder?

Do you see the same issues Skanect reported with surfaces that reflect IR light differently (glossy paint on a flat book cover was the example image given)

[Tobby Ryan]

I only tested to begin with, as I built the rig for the xbox 360 rig for 3d body scans.  I will do a scan later tonite of something black and shiny to see, then post the pictures.  I am curious as well.

[NICE3DLAB Nice]

Hi you all !

I've tried to use Kinect v2 and as I read it doesnt work. Could you anyone tell me how to export the color model from 3D Builder ? I didn't find how to do it. 

And a Have a problem. Since a install the Kinect v2, my Skanect doesnt detect the Kinect v1... I have uninstalled and restalled everything and it doesnt work...

Thanks for helping me ;)

[Tobby Ryan]

Even though this is not a 3d builder forum, exporting in 3d builder as a PLY will keep the color. Opening in meshlab, you can save as OBJ and the color png that goes with it.

In Meshlab, load the PLY
Filters/Texture/convert per vertex uv to per wedge uv
Filters/Texture/Set Texture
Filters/Texture/Vertex Color to Texture
Now you can save the Obj file, and have the colorized obj.png file to fix later. 

You might have to do some clean-up, on the png file.

[CP HO]

Dear Marek

I have tried the 3D builder that comes with Windows 10 but there is NO scan button. Only how to work with fixed objects.

Can I know how can I install the scan function.

Since this is not a 3d builder forum, please send me a reply by private email

cp...@hkma.org

thanks a million

cp

[Sumanth Alwala]

Windows released the standalone 3d scan app for windows 10 recently. You can download it from the Windows app store. 

Any update on Skanect for Kinect v2? 

[Marcus Wolschon]

That only helps for Windows users.

I'm using Skanect on MacOS and my Kinect V2 is collecting dust ever since I got it pre-ordered from Microsoft on the day it was released based on the promist of Skanect to support it when it's avaliable.

--

[Marcus Wolschon]

Did anyone use that new app yet?
The former "3D builder" app created much worse results with a Kinect V2 then Scannect on MacOS with a Kinect V1 despite the better time-of-flight camera
instead of projected IR dots.

[E. Barea Escobar]

hello, 

after one and a half years I just want to know if you are going to give support to kinect v2 at any time.

thank you

[Nicolas Burrus]

Hi, no support for Kinect v2 is planned, since the same issues remain with this sensor for 3d scanning.

--

You received this message because you are subscribed to the Google Groups "Skanect" group.

To unsubscribe from this group and stop receiving emails from it, send an email to skanect+u...@googlegroups.com.

[E. Barea Escobar]

Ok thanks

[Giovanni Polucchio]

Hi, what about using something like Intel RealSense F200. It's frontal close distance sensor with promising quality. I have a copy of skanect but I think realsense would boost the quality.

[gigafaze]

Kinect 2 sensor works with 3D Scan. I have it, have tried it out compared to Skanect and Primesense Carmine. The Kinect 2 captures about half the detail resolution.  

  

A suggestion for Skanect. Add a Height Field Displacement to the Processing. Height Field Displacement from the Texture map would add a huge amount of detail. If the lighting is directly behind the camera, the light field depth with be from the foreground to the background. There are several Height Field Displacement tools out there, MS 3D Print has it as a 2D to 3D object option, there is one in Ideamaker and Shapeways has started one on their website.  https://www.shapeways.com/creator/2dto3d#landing

There doesn't seem to be any scanning software that has thought of doing this in the processing section. The digital photo captures all the detail the scanning hardware doesn't have the resolution or detection to grab. Getting displacement data from the photo image will really enhance the 3D model using software. It is possible to do this manually, take the digital photo, turn to grey scale, invert if necessary, map to the 3D model and displace more detail into the geometry. Very tedious and time consuming though.

[George Profenza]

Just tried Skanect and it's pretty awesome!

After playing with an Asus Xtion sensor on OSX and seeing the resolution I thought of using Kinect v2.

Grabbed, libfreenect2, compiled with openni2 support, added the libreenect-openni2 driver to Skanect's Drivers...and got it to crash:

Device 0:  "GeForce GT 750M"  2048Mb, sm_30, 384 cores, Driver/Runtime ver.7.50/7.50

Invalid version or not a cyphertext.

Invalid version or not a cyphertext.

Invalid version or not a cyphertext.

QUrl("qrc:/qml/skanect.qml")

[Info] [Freenect2Impl] enumerating devices...

[Info] [Freenect2Impl] 8 usb devices connected

[Info] [Freenect2Impl] found valid Kinect v2 @20:14 with serial 500722542042

[Info] [Freenect2Impl] found 1 devices

[Info] [Freenect2Impl] enumerating devices...

[Info] [Freenect2Impl] 8 usb devices connected

[Info] [Freenect2Impl] found valid Kinect v2 @20:14 with serial 500722542042

[Info] [Freenect2Impl] found 1 devices

Invalid version or not a cyphertext.

[Info] [Freenect2DeviceImpl] opening...

[Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 20*16384 ir: 4*128*33792

[Info] [Freenect2DeviceImpl] opened

[Info] [Freenect2DeviceImpl] starting...

[Info] [Freenect2DeviceImpl] submitting rgb transfers...

[Info] [Freenect2DeviceImpl] submitting depth transfers...

[Info] [Freenect2DeviceImpl] started

file::/qml/skan/Skan.qml:320:13: QML QDeclarativeColumn_QML_160: Cannot specify top, bottom, verticalCenter, fill or centerIn anchors for items inside Column

file::/qml/skan/steps/Share.qml:139: TypeError: Result of expression 'export_color_space' [null] is not an object.

Invalid version or not a cyphertext.

Invalid version or not a cyphertext.

file::/qml/skan/steps/Prepare.qml:302:32: QML Selector: Binding loop detected for property "currentIndex"

file::/qml/skan/steps/Prepare.qml:362:32: QML Selector: Binding loop detected for property "currentIndex"

file::/qml/skan/steps/Prepare.qml:396:32: QML Selector: Binding loop detected for property "currentIndex"

file::/qml/skan/steps/Prepare.qml:469:32: QML Selector: Binding loop detected for property "currentIndex"

Invalid version or not a cyphertext.

Invalid version or not a cyphertext.

Invalid version or not a cyphertext.

Invalid version or not a cyphertext.

file::/qml/pad/FittedList.qml:30:5: QML QDeclarativeListView_QML_65: Binding loop detected for property "focus"

file::/qml/skan/steps/Record.qml:118:17: QML Slider: Binding loop detected for property "value"

Invalid version or not a cyphertext.

Invalid version or not a cyphertext.

file::/qml/skan/steps/Share.qml:232:32: QML FancyTextInput: Binding loop detected for property "text"

NEW STEP: 0 / 

[Info] [VTRgbPacketProcessor] avg. time: 17.5833ms -> ~56.8721Hz

[Info] [VTRgbPacketProcessor] avg. time: 17.3661ms -> ~57.5834Hz

fx=1081.37,fy=1081.37,cx=959.5,cy=539.5

fx=365.056,fy=365.056,ix=259.05,iy=207.753,k1=0.0949921,k2=-0.275369,k3=0.0925599,p1=0,p2=0

[Info] [DepthPacketStreamParser] 298 packets were lost

[Info] [Freenect2DeviceImpl] stopping...

[Info] [Freenect2DeviceImpl] canceling rgb transfers...

Segmentation fault: 11

I was just about to post on the forum to ask for support on getting this going,

then I read the above and saw the images. It makes sense :)

[Allan Caddock]

why l getting only 1/2  object  all the time