Read QR Code & Have a marker

[thomen]

Hi all i was looking at the augmented business card posted on t-o-x-i-
n.de and was wondering how one would do something combining a marker
and a qr code..
what id ultimately like to do is show the cam a qr code and based on
that display certain content on the marker ie a different pv3d dae or
load a different xml file etc..

any ideas on how the 2 can co-exist?

Cheers,
Tom

[Makc]

someone already did that in japan few months ago.
don't have a link at hand, but I'm sure it's googleable.

[jbyrne2007]

http://www.encyclopedia.com/video/I6MiJKgtbug-qr-code-ar-demo-augmented.aspx

This the one Makc?

[Makc]

hard to say, I saw it linked on twitter, I guess, read briefly and
never bookmarked

[thomen]

I guess I was looking for some source examples as well.. on the whole
detect and then load object based on detection etc..

has anyone got any of that sort of source available?

[Almog Koren]

I have source available for the QA Maker however its not implanted into AR.

  

Almog Design
Mobile:  +972 54 663 3308
Email:    al...@almogdesign.net
Site:       www.almogdesign.net
Twitter:   @almogdesign

[jbyrne2007]

Where is it available at?

> > > never bookmarked- Hide quoted text -
>
> - Show quoted text -

[Almog Koren]

I need to uploaded, I will send it to your email address.

Almog Design
Mobile:  +972 54 663 3308
Email:    al...@almogdesign.net
Site:       www.almogdesign.net
Twitter:   @almogdesign

[thomen]

I was just thinking about it all as the guy from t-o-x-i-n.de used
something similar for his augmented business card and was wondering
how it was done

On Oct 22, 9:56 pm, Almog Koren <al...@almogdesign.net> wrote:
> I need to uploaded, I will send it to your email address.
>
> Almog Design
> Mobile:  +972 54 663 3308
> Email:    al...@almogdesign.net
> Site:      www.almogdesign.net
> Twitter:   @almogdesign
>

> On Thu, Oct 22, 2009 at 10:40 AM, jbyrne2007 <jbyrne2...@hotmail.co.uk>wrote:
>
>
>
>
>
> > Where is it available at?
>
> > On Oct 21, 11:54 pm, Almog Koren <al...@almogdesign.net> wrote:
> > > I have source available for the QA Maker however its not implanted into
> > AR.
>
> > > Almog Design
> > > Mobile:  +972 54 663 3308
> > > Email:    al...@almogdesign.net
> > > Site:      www.almogdesign.net
> > > Twitter:   @almogdesign
>
> > > On Thu, Oct 22, 2009 at 12:52 AM, thomen <penny.lane.m...@gmail.com>
> > wrote:
>
> > > > I guess I was looking for some source examples as well.. on the whole
> > > > detect and then load object based on detection etc..
>
> > > > has anyone got any of that sort of source available?
>
> > > > On Oct 22, 6:40 am, Makc <makc.the.gr...@gmail.com> wrote:
> > > > > hard to say, I saw it linked on twitter, I guess, read briefly and
> > > > > never bookmarked- Hide quoted text -
>

> > > - Show quoted text -- Hide quoted text -

[eric socolofsky]

he didn't use the QR code for tracking, he just used it to provide additional information.  his thinking was that the QR code could be swapped out with the business card information, while the FLAR marker would be used for the AR content (model transformation).

if i remember correctly.

i believe he's on this list, anyway, he promoted the project here, so you could always search the archives and contact him directly.

-eric

[toxin]

hey thomen,

all the tracking is done with the FLARmarker on the business card. the
printed QR code on the card contains the URL of the presentation
xml .. where for example all the assets of the porfolio presentation
are stored.

On 23 Okt., 02:24, eric socolofsky <e...@transmote.com> wrote:
> he didn't use the QR code for tracking, he just used it to provide
> additional information.  his thinking was that the QR code could be swapped
> out with the business card information, while the FLAR marker would be used
> for the AR content (model transformation).
>
> if i remember correctly.
>
> i believe he's on this list, anyway, he promoted the project here, so you
> could always search the archives and contact him directly.
>
> -eric
>

[guillermo]

Hello Guys,

for long time I was too interested in read QR code parallel wit AR
Marker.

I tried the the implementation from libspark but is not 100% ready,
the detection of the QR-Code.

I saw too the idea from toxin, but I think was just one idea Ar
Business Card, because since July the project don't have any progress.
He promess to open as Open Source. I would like to know waht he think
about it.

By the way the mix between AR-Marker and QR-Code is interesting but
the implementation is really until now no easy.

Best,

Guillermo

[dieGopen.com]

Hola Guillermo,

I'm working on that direction too. AR + QR will rule the world! If I do any progress I'll let you know :)

Diego

[guillermo]

Grazie Diego,

Guillermo

[L]

For using QR code in AR, I only see one for this goal.

QR Code AR Demo ( Augmented Reality )
http://www.youtube.com/watch?v=I6MiJKgtbug

Sorry, not help you much for useful information.

[Mikael Emtinger]

Hey!
I've written a new matcher for FLARToolKit to detect 4x4 ID-markers
(basically a 16 bit number). It's not 100% done yet (some cleaning up
to do and some more tests) but I will probably release it through
Saqoosha any week now. It let's you have something like 10000
different IDs in your library, which usually should be enough. It's
also a bit faster than the current compare algorithm. You use the ID
to pick whatever content you like to use from your database.

The main problem with ID- and QR Code-recognition is to be certain
about what is actually on the marker. If one "bit" is uncertain, it
changes the information drastically. In my 4x4 ID-marker recognition
algorithm I've "solved" this by letting you train the algorithm with
IDs that you're looking for, and these get prioritized in the list of
possible result (which could be quite many in bad lighting conditions
and/or steep angles). The problem still remains if you use two IDs
that look similar, so you need to pick them carefully.

Making a QR Code-matcher shouldn't be too difficult to implement if
you manage to solve the uncertainty problem explained above. My guess
is that you need to use quite high resolution (both on the raster you
send into FLAR and on the picker / matcher) and "train" it as I do.

Hopefully I will be able to put together the new 4x4 ID classes and a
example project soon.

/m

On Oct 28, 8:28 am, L <lcliu2...@gmail.com> wrote:
> For using QR code in AR, I only see one for this goal.
>

> QR Code AR Demo ( Augmented Reality )http://www.youtube.com/watch?v=I6MiJKgtbug

[eric socolofsky]

sounds interesting...will this just be an alternate setting in
FLARToolkit, like patternType or something?

i'm a bit confused; if you have a limited number of patterns your
application is looking for, why do you need to train the algorithm?
wouldn't it just weight matches against the list of patterns (e.g.
FLARCodes) you load into FLARToolkit? or is the idea that with this
setup, you don't need to load .pat files, you can just show an app a
4x4 marker and it will spit out an ID? in either case, i guess i'm
not clear on where/how the 'training' takes place.

tell us more! sounds great.
-eric

[Mikael Emtinger]

That's right, you don't need FLARCodes at all - the algorithm reads
the 16-bit number (or ID) that is printed on the marker. And returns
it, together with a confidence. You use that number to display
something on the marker.

The problem is at steep angles, when moving the marker fast or during
bad lighting conditions. The number then becomes hard to read and the
algorithm returns a false ID. To get around this you can, but you
don't have to, tell the algorithm at init which IDs you use in your
application. The algorithm puts these prioritized IDs in the beginning
of the list of possible results (it returns a result for each possible
ID, sorted in order of confidence).

That said, I will take the algorithm for another swing and let the
picker pick at higher resolution (today it's just 4x4 pixels, I'll try
something like 64x64 pixels) and see if it's possible to make better
guesses with more pixels to look at. This will not change the number
of IDs the algorithm can return, it's only internal. With more pixels
to look at I think I better can determine if, for example, the marker
is motion-blurred (moved fast) etc.

Ontop of this you can do a slight rewrite of FLARManager to help you
get a more stable result.

/e

[eric socolofsky]

if your version can return, every frame, a list of possible matches hashed by location, a wrapper like FLARManager can check the list and make an educated guess based on the pattern ID of a marker that was previously at that location.  rough code:


// all IDMarker possibilities, hashed by Point locations
// e.g.:
//    {
//        new Point(10, 20) : Vector.<IDMarker>(new IDMarker(7), new IDMarker(9)),
//        new Point(150, 210) : Vector.<IDMarker>(new IDMarker(3), new IDMarker(14)),
//    }
var markerPossibilitiesByPoint:Dictionary = markerDetector.getIDMarkers();

var idMarkersAtPoint:Vector.<IDMarker>;
var idMarker:IDMarker;
var activeMarker:FLARMarker;

for (var idMarkerLocation:Point in markerPossibilitiesByPoint) {
    for (var i:int=0; i<activeMarkers.length; i++) {
        activeMarker = activeMarkers[i];
        idMarkersAtPoint = markerPossibilitiesByPoint[idMarkerLocation];
        if (Point.distance(activeMarker.centerpoint, idMarkerLocation) < this.markerUpdateThreshold)) {
            for (var j:int=0; j<idMarkersAtPoint.length; j++) {
                idMarker = idMarkersAtPoint[j];
                if (idMarker.id == activeMarker.id) {
                    // FOUND MATCH;
                }
            }
        }
    }
}

does that make sense?  is that something that could fit with the logic you're working with right now?  also, your IDMarker class (or whatever you call it) could store the confidence value of the match as well.  in fact, it might as well store the location, then getIDMarkers wouldn't necessarily have to return a hashtable, it could just return a list.

point is, if you return multiple id possibilities at each location a marker is detected, a wrapper could make an educated guess as to what id the marker actually is, based on the id of a marker that was close to that position last frame.

-eric

[Makc]

Can anyone tell me why current flartoolkit is not enough to do this?

There are three square things in every QR code that can be recognized
as flartoolkit markers, right? This should actually give you three
times more stable 3D location than just one normal marker. Using this
information, you can easily perform homography and then read QR code
data using simple getPixel loop, but if you use existing QR reader
this is not necessary.

Maybe I am missing something in big picture, but this seems like a day
of work for someone like Eric.

[eric socolofsky]

heh. thanks for the nomination, makc.

i think the tricky thing here would be recognizing the markers at that
small a size.

however, it might not be that tricky, since the content of those
markers doesn't matter, just their location.

could be a fun project, would probably only be a day of work for
someone like makc ;)

[Makc]

are you challenging me for proof-of-concept demo? what do I win with it?

[eric socolofsky]

my undying love and affection :)

[Mikael Emtinger]

That's a very good idea. I could easily implement your idea under the
hood so you don't have to care about it. I'm sure it'll be much more
stable.

I was thinking about implementing something similar to speed up the
labeling process - basically use last frame's squares and have that as
a starting point for the current frame's labeling. Another idea that I
might look into, concerning the labeling, is to perform that in a
lower resolution. As far as I've understood, the purpose of the
labeling is to find the squares. This could probably be done in a
lower resolution (ie. 160x120) and then scale up the square values to
the original raster resolution. The labeling is a huge bottle-neck.

Thanks for the input!
/m

> <mikael.emtin...@gmail.com>wrote:

[makc]

done
http://makc3d.wordpress.com/2009/10/30/augmented-reality-and-qr-codes/

where do I get my prize now?

[eric socolofsky]

nice!

you definitely win the prize. most impressive.

[eric socolofsky]

the purpose of labeling is to find candidates for outline detection --
it's prep work to determine areas ("labels") that could possibly be
(distorted) square outlines.

as far as i can tell, it's currently the slowest part of the whole
engine.

FLARManager has a hook to speed things up by specifying a minimum size
for labeled areas (minimumLabelSize), which helps somewhat, but
increasing that number too much reduces detection at smaller sizes.

the idea of using a lower resolution for the labeling could be a good
one; it will have the same issue that minimumLabelSize has, which is
that it will work poorly for smaller (i.e. further-away) markers. but
you're right, the square outline doesn't need as high resolution for
accurate detection as the pattern within the outline.

i get the feeling you're already pretty up on this stuff, but in case
you want verification of your own findings, you can take a read
through this article i wrote:
http://www.insideria.com/2009/07/flartoolkit-and-flarmanager.html
that describes some of the processes within FLARToolkit (and
FLARManager).

looking forward to see what you put together! and if you need a
second pair of eyeballs (or hands on a keyboard), let me know...
-e

[makc]

yeah, now someone needs to re-do this using 2D points from squares,
because 3D sucks (too much error)

[thomen]

wow awesome poc!

it'd be interesting to compare the performance vs the 16 bit number
algorithm as suggested above

On Oct 31, 1:39 am, makc <makc.the.gr...@gmail.com> wrote:
> donehttp://makc3d.wordpress.com/2009/10/30/augmented-reality-and-qr-codes/

[Mikael Emtinger]

@eric: Oh! Didn't realize you wrote the FLARManager - great job! I did
the GE-site. I'll let you know as soon as I need an extra pair of
hands/eyes.

Anyway, I really hope to get some time soon (awaiting a project GO!)
to get the 4x4-ID-classes cleaned up, as well as looking into
optimizing the labeling (the lower resolution solution). I'll also try
and implement the ID-recognition into the Alchemy-version (got word
from Nyatla yesterday that it's updated, haven't had time to download
and compile yet).

And, going back to this thread's headline - it's not that hard to
implement a QR-detector directly into FLARToolKit. All you need to do
is to write a new matcher and have a pretty high resolution on the
picker (I'd say 64x64 pixels at least). It should be much faster than
tracking three codes. But I am, as Eric, very impressed with the QR-
demo - great job!

/m

[Makc]

> The main problem with ID- and QR Code-recognition is to be certain
> about what is actually on the marker. If one "bit" is uncertain, it
> changes the information drastically.

have you thought about using correction codes? you would have much
less possible numbers to store, but much better error resistance.

> And, going back to this thread's headline - it's not that hard to
> implement a QR-detector directly into FLARToolKit. All you need to do
> is to write a new matcher and have a pretty high resolution on the
> picker (I'd say 64x64 pixels at least). It should be much faster than
> tracking three codes.

that would not really work out of the box because qr code will be
broken into many squares. if you could guarantee solid black frame
around qr code - then yes.

the same way, you could detect arbitrary jpegs, for example, but the
problem is, again, that you can't force black square frame.

> it'd be interesting to compare the performance vs the 16 bit number
> algorithm as suggested above

obviously you can't compare them because 4x4 is not enough to fit qr code :(

[Mikael Emtinger]

> have you thought about using correction codes? you would have much
> less possible numbers to store, but much better error resistance.

With "correction codes" do you mean something along the lines of a
checksum built into the 4x4ID? If that's what you mean, it's a very
good idea. I'll see if I can sort something out.

> that would not really work out of the box because qr code will be
> broken into many squares. if you could guarantee solid black frame
> around qr code - then yes.

Of course, you need to have the QR-code within a black frame (so it
wouldn't work with just any QR-code).

I've just switched my working environment and is back on track. I'm
starting to clean up the code for a pre-release as we speak.

/m

[Makc]

On Fri, Nov 6, 2009 at 4:21 PM, Mikael Emtinger
<mikael....@gmail.com> wrote:
>
>> have you thought about using correction codes? you would have much
>> less possible numbers to store, but much better error resistance.
>
> With "correction codes" do you mean something along the lines of a
> checksum built into the 4x4ID? If that's what you mean, it's a very
> good idea. I'll see if I can sort something out.

not just checksum but codes that allow to recover from error, see
http://en.wikipedia.org/wiki/Error_correcting_code

[eric socolofsky]

can you elaborate on how error correction codes might work in the
context of a pattern matcher? read thru the wikipedia entry and
related entries, but it's a bit over my head....

re: using a checksum, error correction could essentially happen over
time, instead of within one frame. something like if (checksum != 0)
{ use last frame's pattern ID }. it's unlikely (near-impossible for
most applications) that a pattern ID would change from frame to frame
if it's in the same location.

[eric socolofsky]

oh, and mikael -- nice to meet you! congrats on the GE project...i
imagine a lot of the folks on this forum (myself included) would not
be here if it wasn't for that project. or, at least, it would have
taken us a while longer to get here....

and, fwiw -- exciting to see some interesting conversations about
"under the hood", been looking forward to this :) if only i had more
time to tinker.

-e

[Makc]

On Fri, Nov 6, 2009 at 8:39 PM, eric socolofsky <er...@transmote.com> wrote:
>
> can you elaborate on how error correction codes might work in the context of
> a pattern matcher?  read thru the wikipedia entry and related entries, but
> it's a bit over my head....
>
> re: using a checksum, error correction could essentially happen over time,
> instead of within one frame.  something like if (checksum != 0) { use last
> frame's pattern ID }.  it's unlikely (near-impossible for most applications)
> that a pattern ID would change from frame to frame if it's in the same
> location.
>

basically these codes work as follows: you have, for examle, 8 bits of
input; you add another 8 bits calculated by the algorithm; now if you
alter ANY 1 or 2 bits in these 16, you can run the algorithm, detect
the change and restore original bits. exact number of bits you can
change before error is irreversible depends on algorithm and, I think,
is limited by some theorem.

in context of 4x4 markers, the above scenario means you limit a total
number of markers to 2^8 instead of 2^16 but you gain far better
stability of readings between frames.

I still have one such algorithm described in details in my old notes,
I might try to implement it some day for kicks.

[Mikael Emtinger]

Strange, sent a message from my iphone that doesn't show up here.
Anyway...

Spent the better part of my friday (read: evening) implementing 4x4ID
markers with parity bits error detection. Basically I use 3x3 bits for
the ID (not all numbers can be used, though, you'll see why) and 3x3
parity bits (3 for horizontal, 3 for vertical). This allows me to
correct when the number of false parity bits are an even number and,
when the number is >2, only when the parity bits don't overlap. But it
shows that this doesn't help much - when there's an error, it's quite
a sever one. Usually the errors are results of motion-blur, no code
correction algorithm in the world could correct that...

I think Eric is right, it's better to check positions between frames.
Another solution might be to pick more than 4x4 pixels (as I do now)
and try to figure out if there's a motion-blur going on and try to
cook down it down to the 4x4 pixels we need.

That said, it's not all bad with error detection - the result is
stable (when readable) and you don't have to feed FLAR with what
markers you have in the application, as you had to before. Both
algorithms are available - I haven't compared stability nor speed,
please feel free.

I'll put together a library tomorrow for you to look at. Not sure how
it is supposed to be shared - is it ok if I post a link here? I've
already sent an early demo to Saqoosha (weeks ago) and will ofcourse
mail him tomorrow's library and let him know I've posted a link here.

/m

On Nov 6, 8:41 pm, Makc <makc.the.gr...@gmail.com> wrote:

[eric socolofsky]

how, with only 3x3 resolution, do you deal with marker orientation?
there aren't many non-symmetrical options at such a low res...even at
4x4 there aren't very many options.