ThreePhase C# Port
fisj
I've been porting Kyle's code to c#, and it looks like I've run into
some tearing issues when unwrapping the phases.
Binaryillenium suggested the following:
"One method to reduce the tearing that works decently (though is
costly in capture time) is to do at least three passes at different
frequencies, and take the median of the unwrapped phases.
One thing I haven't tried (for a single pass) would be to 'seed' the
phase unwrapping at different locations and not just the center,
though that are still on the target (and not on masked off areas), and
take the median of those.
Another thing to try is to have three variations on the standard flood
fill, ones that for example fill more preferentially in different
directions, and median the results.
You might want to move discussion over to http://groups.google.com/group/structured-light.
Also share a link to your source. Also if you want you could probably
host it within the Structured Light google code project."
My response:
By frequencies I assume you mean the individual RGB channels. My usual
trick is to take the Luminance, (R*0.3 + G*0.59 + B*0.11) and run that
through. I also tried (R+G+B/3.0) but got the same artifacts.
Just now, I tried phase wrapping each channel, and then averaging
them, and still no dice - actually worse. I'll look into more robust
floodfill algorithms later today.
Here's the link for a scan I did with the tearing artifact
http://tinypic.com/r/vimm8y/6
Here's the link for kyle's own test subject: http://tinypic.com/r/2vm5yqw/6
(the tearing happens different ways because I was messing the the
order of the flood fill)
I'll post the code when I clean it up a bit
Thanks!
-fisj
Kyle McDonald
being too low. In the image of the 'test subject' (my girlfriend) I
provided, the background should all be black. If it isn't black, that
means that the algorithm is trying to propagate across noisy regions,
leading to these glitches.
By different frequencies, BinaryMillenium is referring to using
different three phase patterns. The ones I provided are something like
20 lines over the height of the screen. If you use more than just those,
you can get different resolutions of depth data. This will avoid
glitches at large depth discontinuities, but it won't necessarily
resolve the issue you're showing.
There are some versions of flood fill that are only slightly more
complex than the one I provided. For example, before propagating at a
single pixel, figure out which direction has the least noise and do that
first. Or, keep the "to process" list in a priority queue where pixels
you have more confidence about are closer to the front.
I also pointed fisj to a post I made last week that I have yet to
implement, but is a lot more complicated:
http://erraticsemaphore.blogspot.com/2010/01/continuity-guided-2d-phase-unwrapping.html
Kyle
--
I tweet every 140 characters I type @keytweeter
fisj
@Kyle: I'll fiddle with the noise thresholds some more. I can
certainly mask off areas, even use something like green screen.
However my main confusion is the code is as close as I could get to
yours, and I was expecting near identical results. I'll keep prodding
the code until it works :). Thanks for the blog link, I'll be sure to
read it tonight.
I've popped my src and release here: http://dl.dropbox.com/u/2770795/cs_threephase.rar
You can input varying noise thresholds in the command line. This src
won't be there forever, its just a temporary place to host this. You
will probably need Microsoft .NET Framework 3.5 installed to run it.
Oh, one qustion I did have. The wavy phase line artifacts that show up
in all the scans I've seen ... are those because the algorithm assumes
linear intensities? If so, you could just use linear tifs saved from
RAWs. That, or if you know the camera response function of your camera
you can map the intensities back using a polynomial curve or LUT. I
should really read the original paper in more detail. :\
I'll keep you guys posted if I find anything regarding the tearing
artifacts.
-fisj
On Jan 31, 5:20 pm, Kyle McDonald <k...@kylemcdonald.net> wrote:
> Yes, I'm almost positive this is just an issue with the noise threshold
> being too low. In the image of the 'test subject' (my girlfriend) I
> provided, the background should all be black. If it isn't black, that
> means that the algorithm is trying to propagate across noisy regions,
> leading to these glitches.
>
> By different frequencies, BinaryMillenium is referring to using
> different three phase patterns. The ones I provided are something like
> 20 lines over the height of the screen. If you use more than just those,
> you can get different resolutions of depth data. This will avoid
> glitches at large depth discontinuities, but it won't necessarily
> resolve the issue you're showing.
>
> There are some versions of flood fill that are only slightly more
> complex than the one I provided. For example, before propagating at a
> single pixel, figure out which direction has the least noise and do that
> first. Or, keep the "to process" list in a priority queue where pixels
> you have more confidence about are closer to the front.
>
> I also pointed fisj to a post I made last week that I have yet to
> implement, but is a lot more complicated:
>
> http://erraticsemaphore.blogspot.com/2010/01/continuity-guided-2d-pha...
>
> Kyle
>
>
>
>
>
> fisj wrote:
> > BinaryMillenium suggested I post here, so to catch the reader up ...
>
> > I've been porting Kyle's code to c#, and it looks like I've run into
> > some tearing issues when unwrapping the phases.
>
> > Binaryillenium suggested the following:
>
> > "One method to reduce the tearing that works decently (though is
> > costly in capture time) is to do at least three passes at different
> > frequencies, and take the median of the unwrapped phases.
>
> > One thing I haven't tried (for a single pass) would be to 'seed' the
> > phase unwrapping at different locations and not just the center,
> > though that are still on the target (and not on masked off areas), and
> > take the median of those.
>
> > Another thing to try is to have three variations on the standard flood
> > fill, ones that for example fill more preferentially in different
> > directions, and median the results.
>
> > You might want to move discussion over tohttp://groups.google.com/group/structured-light.
fisj
I was using a stack array for the floodfill, which was using the exact
wrong ordering. This resulted in a scanline-like sweep, which picked
up nasty errors on the borders of mask.
As best I can tell the use of the luminance value instead of the
"(color1 & 255) / 255.;" is actually better. There seems to be less
noise, and less prominent band artifacting. This code also writes out
16bpp tiffs, though its by no means interactive or fun. :)
I may take a crack at the quality guided phase unwrapping, in the
future. I'll let yas know if I do.
Thanks for the help guys!
-fisj
Kyle McDonald
> The wavy phase line artifacts that show up in all the scans I've
seen... are those because the algorithm assumes linear intensities?
Yes, this is basically what's going on. There are a lot of things going
into the nonlinearity of the reflected light intensity, but modeling it
with a LUT is a great approach. Polynomial curve is probably overkill.
> luminance value instead of the "(color1 & 255) / 255.;"
I totally forgot about this, the & is from when I was still using
grayscale images and it was faster than brightness(). You're exactly
right about luminance being better.
> quality guided phase unwrapping
I just posted an implementation:
http://code.google.com/p/structured-light/source/browse/#svn/trunk/Processing/ThreePhasePriority
Instead of the usual flood fill unwrapping pattern:
http://www.flickr.com/photos/kylemcdonald/4321065737/
It produces a much more reasonable pattern:
http://www.flickr.com/photos/kylemcdonald/4321761834/
It's idea of "quality" is based on a per-pixel metric of "phase range":
http://www.flickr.com/photos/kylemcdonald/4320973819/in/photostream/
Basically, if something is less reflective it's going to be more noisy.
A contextual, local metric (e.g., the difference between the current
phase and previous phase) would probably be better, but for some reason
I couldn't get it to work. It would be slightly more expensive to
compute, but as long as we're not talking about real time decoding here
it's still not an issue.
Kyle
--
Kyle McDonald
unwrapping and got some good results.
http://www.flickr.com/photos/kylemcdonald/4321872724
You can see how complex the unwrapping behavior is -- it responds to
smooth vs rough textures (local pixel quality) as well as dark vs light
areas (phase range).
Also, it looks like the priority queue never gets bigger than .7% of the
image, which is great for speed.
If you anyone is interested in this more accurate unwrapping technique,
could you look over it:
http://code.google.com/p/structured-light/source/browse/#svn/trunk/Processing/ThreePhasePriority
If it looks right, and the speed is comparable to flood fill in C++, I
might try decoding some problematic video sequences.
Kyle
fisj
I started to port your new changes, then realized I'd never dealt with
PriorityQueues or implements, realized I'd probably have to roll my
own queue, then motivation slowed. Maybe later this week.
Overall it looks like the areas that it does unwrap, it unwraps at a
better quality, but isn't necessarily more robust overall. Thanks for
the example!
-fisj
On Feb 1, 3:38 am, Kyle McDonald <k...@kylemcdonald.net> wrote:
> I couldn't give it up, so I just rewrote the pixel quality guided
> unwrapping and got some good results.
>
> http://www.flickr.com/photos/kylemcdonald/4321872724
>
> You can see how complex the unwrapping behavior is -- it responds to
> smooth vs rough textures (local pixel quality) as well as dark vs light
> areas (phase range).
>
> Also, it looks like the priority queue never gets bigger than .7% of the
> image, which is great for speed.
>
> If you anyone is interested in this more accurate unwrapping technique,
> could you look over it:
>
> http://code.google.com/p/structured-light/source/browse/#svn/trunk/Pr...
Kyle McDonald
to avoid propagating across noisy pixels until it's done with better
candidates. For example, using a noise threshold 0, a comparison between
flood fill and priority queues:
http://www.flickr.com/photos/kylemcdonald/4323603585/
The problem remains: sometimes the shortest path from one pixel to
another is not the "correct" path. It's more like: out of all the
possible paths from one pixel to another, the most common/mode of all
the unwrapped phases is the correct one. This is hard to optimize...
Fortunately this shouldn't be too hard to port to C++ as the STL has a
priority_queue class. But I can't say the same for C# :(
Kyle
fisj
the unwrapping algorithms. Yup, you're right, there is a significant
reduction in noise, and completeness of the reconstructed mesh. See
below:
http://dl.dropbox.com/u/2770795/tangarine_quality.jpg
I did notice something interesting when I compared the two
reconstructions against each other. The phasepriority code seems to
have radically displaced the tangarine at the bottom closer to the
camera than it actually was. Example link below:
http://dl.dropbox.com/u/2770795/tangarine_offset.jpg
http://dl.dropbox.com/u/2770795/img.rar
The original floodfill code placed it correctly. I included a link to
the .rar containing the original input images. Its got me a little
puzzled as to why it might do that. Perhaps because of the
discontinuity between the tangarine and the shoe, it has skipped a few
phases up and has displaced its depth forward? I was wondering if you
had any insights.
Oh, and I wanted to thank you for all your hard work on this. This
stuff is super interesting.
-fisj
Kyle McDonald
Looking good!
> puzzled as to why it might do that. Perhaps because of the
> discontinuity between the tangarine and the shoe, it has skipped a few
> phases up and has displaced its depth forward? I was wondering if you
> had any insights.
Yes, it's because of the discontinuity. In general, if there is a shadow
or a depth discontinuity of greater wavelength of the three phase
pattern at that depth, then relative position of two disconnected
components is not well defined. In other words, both positions are
equally "correct" unless you have more information (lower frequency
pattern, or multiple perspectives for example).
> Oh, and I wanted to thank you for all your hard work on this. This
> stuff is super interesting.
I've just been getting things rolling, it's everyone else that's using
and developing it that is really pushing it along :) Thanks to
BinaryMillenium, Eric Koehler, Roy MacDonald, PRICKIMAGE, etc.!
zxl...@gmail.com
fisj
--
You received this message because you are subscribed to the Google Groups "structured-light" group.
To unsubscribe from this group and stop receiving emails from it, send an email to structured-lig...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.