[openARweb] FPML :-) Focal Plane Markup Language

[MikeLiebhold]

changing the subject for a minute:

Before we invent something totally new, :-) why don't we look at some
of the smaller pieces right here on the buffet of ideas:

Let's pretend we can design one first element of an open AR stack:
e.g. a FPML :-) Focal Plane Markup Language for a web browser for
the real world.

Use case: How do i display and interact with the bits on my open ARweb
client -focal plane- view, ( on my hand held, or in my glasses)
interacting with data from the whole world wide web? -

some possible elements:

1.) APIs for to hardware and search for precise 3d geo position
( field of view, pitch and yaw) with extensible support for rendering
and full web interaction within [ at least] four views:
-rectangle.
-panoramic
-spherical
-stereo

2.) software APIs to query camera hardware and net servers for full
zoom capability to align AR data objects, from a moving frame of
view.. As a query constraint, and filter; selectively disclosed to
external AR web servers. - Support for authentication)

[imho: KML+OGC are about 62% the way there as view/ interaction
language] ... need lots more work... but is open enough to be doable]

3.) support for a rich, secure client-side computing environment with
dynamically imported decoders and renders for AR media objects,

[ imho: HTML+XML, AJAX/JSON/DOM are about 68% there for AR, with lots
of room for grass roots AR development - (inevitable)


????

[Tish Shute]

Yes, this idea of looking at a FPML does seem a good smaller piece that we could come together productively on.
T 

--
Tish Shute
Founder
Ugotrade

http://www.ugotrade.com
Register for Augmented Reality Event at http://augmentedrealityevent.com/register
tish....@gmail.com
@tishshute
skype tish.shute
+1 646 753 0539

[Tish Shute]

It would be very interesting have Blair MacIntyre and Georgia Tech team in this discussion as they have produced a web standards based AR browser.
T

On Mon, May 3, 2010 at 11:52 PM, MikeLiebhold <mikeli...@gmail.com> wrote:

[Blair]

Ok, hi Tish. :)

Thanks for sending me the pointer, I'll get some of the other folks on
here too. It'll be a little while before I can wade through all the
messages and make any useful contributions.

But, at a high level, as Tish says, we've created an AR browser
(currently on the iPhone, hoping to port to Android and others soon).
It's based as much on existing standards and tech as possible. We've
extended KML as the basic structure, and use HTML5 and javascript (and
soon, Collada) for content and client-side scripting.

We will hopefully present something on it at the AR Event, and hope to
have it in the App Store in the next month (for free, for everyone to
use). We're setting up a set of community web sites, examples of how
to generate channels using various web-programming approaches, and so
on!

On May 4, 10:29 am, Tish Shute <tish.sh...@gmail.com> wrote:
> It would be very interesting have Blair MacIntyre and Georgia Tech team in
> this discussion as they have produced a web standards based AR browser.
> T
>

> tish.sh...@gmail.com

[ThomasWrobel]

Blair - Great to hear from you! *Really* looking forward to seeing
that browser.
I agree there's quite a lot of posts to wade though here, but some
useful stuff said by many people.
Just for clarification ,do you mean your using HTML5 to link to KML
and JavaScript to manipulate it?
Or do you mean your rendering some HTML5 in 3d scenes?
---

Mike - Good idea. After all, the data we send is quite separate to the
methods used to send it. The idea of having AR content expressed as a
sort of mark-up language is quite a valid one. (although would have to
complete against ARML as a proposal)

I will need a few clarifications on your points though, to make sure
we are on the same page here.

" with extensible support for rendering
and full web interaction within [ at least] four views:
-rectangle.
-panoramic
-spherical
-stereo "

Do you mean the existing (2D) Web here? That is, ways to display
traditional web rendered in your hub?
I can picture rectangle and panoramic, but not stereo and spherical?
What would be the visual result of those modes.

Also; Would this not be partly down to the client itself, rather then
in the markup? (I mean, the markup could specify something like the
"@media ARphone" or "@media HMD" ?). But clients might have their own
preferences. For instance a HMD might display a webpage as a small
page, but if the user selects it, it could surround them. A bit like
mobile web browsers today often display the whole page, and then let
you zoom into a wrapped paragraph.

Of course, I could have got the wrong end of the stick here, and your
not talking along these lines at all.

"3.) support for a rich, secure client-side computing environment
with
dynamically imported decoders and renders for AR media objects, "

When you say renderer do you mean for specific media formats, or for
rendering the scene as a whole? (ie, the actual 3d-rendering)
The first seems quite possible, but the second I'm not so sure is
practical.
I think the scene-rendering itself would be too tied to the client/
platform, especially if the client is displaying multiple sources in
the field of view at the same time.
Limited resources on mobile devices would also limit options somewhat.


-Thomas

[Mike Liebhold]

Hi Thomas,

I'll attempt to answer your questions interleaved below.

On 5/4/10 2:11 PM, ThomasWrobel wrote:
> I can picture rectangle and panoramic, but not stereo and spherical?
> What would be the visual result of those modes.
>

I am imagining that a head-mounted display (or contact lenses - on 10
years or so) could cache a continuous display that extends 360degrees
around the frontal field of view, so that if you move your head, the
data would appear stable, with no network querie for refresh,. This
implies a spherical projection for rendering, rather than a typicl
rectangular view on a mobile viewfinder. Agian, this need not be
probable in the near term, but any markup language should be extensible
to support this view later on, wiht no need for a completely new markup.
Similarly it would be ideal to implement a semantic framework that will
support stereo3D ealrier than later. I can imagine many home
entertainment use cases for full stereo AR.

> (I mean, the markup could specify something like the
> "@media ARphone" or "@media HMD" ?). But clients might have their own
> preferences. For instance a HMD might display a webpage as a small
> page, but if the user selects it, it could surround them.
>

Yes. I am assuming that any server will be able to to a standard
client-detect, and, if available serve an appropriate experience for the
detected device.

> "3.) support for a rich, secure client-side computing environment
> with
> dynamically imported decoders and renders for AR media objects, "
>
> When you say renderer do you mean for specific media formats, or for
> rendering the scene as a whole? (ie, the actual 3d-rendering)
> The first seems quite possible, but the second I'm not so sure is
> practical.
> I think the scene-rendering itself would be too tied to the client/
> platform, especially if the client is displaying multiple sources in
> the field of view at the same time.
>

This is a probably great unsolved problem. HTML5 allows dynamic
imported decoders using XML, eg. KML ( google earth embedded in a web
page) digital audio, digital video, and 3D, and ray-traced graphics. As
far as I know there's no reason different codecs or renderers can't be
downloaded, and invoked simultaneously. But, Honestly I don't know the
answer for sure, and would appreciate any expert commentary.

- M.

[Blair]

I mean that the 3D scene has HTML5 elements (e.g., whatever you could
put in a <div>) out in 3D, and those elements can have javascript in
them; in effect, imagine something like each KML placemark being a
fully javascriptable HTML5 element, where those 2D elements can either
be screen oriented (e.g., like the little info bubbles in many AR apps
right now) or positioned in full 3D (as a billboard).

So, you can have an interactive HTML5 element on the ground, floating
above a location, or on the side of a building.

Is that what you're asking?

[Anselm Hook]

The thought of a focal plane markup language is interesting to
speculate about. I agree of course we will need some way to define how
to present even the mundane aspects of an augmented reality display.

One note is that since we're running on more limited hardware at the
moment (iphone android etc) supporting many many grammars may be
expensive.... But I agree it is probably inevitable though that AR
browsers will have to support HTML5 + Canvas + Javascript ( and at
some point Canvas will likely include 3d so elements can pop out of
the billboard presumably ) .

[ By an AR browser my mental model is say a mobile phone with gps,
orientation sensor, tilt sensor, camera - or a HMD with a fast refresh
rate - and possibly peeking at the world to get exact orientation and
position (by looking at geometric features in the world and fine
tuning exact focal view based on that). And I imagine that the view is
some kind of synthesis of dynamically generated 3d content blended
into the real world ( perhaps with occlusion culling given an
understanding of where buildings and other obstructions are ) or just
a purely synthetic view registered against the real world quickly
enough that one doesn't get nausea or run into things. ]

Anyway for purely billboards - I suppose that HTML with a bit of
additional information about position and orientation is all we need.
I do kind of wonder if it isn't like the migration from theater to
cinema - and if we're transporting a concept that won't be used much?
It might need to be tested to see what users desire most - the
comfortable or the suitable - or a transition bridge between them?

I would prefer to interact with a small 3d object proximate to me more
so than a billboard. For example I would like to see a restaurant menu
expressed as say a sun with a number of planets orbiting it... rather
than as a list view.

I do assume billboards will be a part of AR - but it makes me wonder
if we even have a design grammar for describing how things are
positioned near to each other in 3d.

In design - the ability to have blocks that pack next to each other in
a pretty and aesthetically pleasing way. We see this in everything
from fonts to CSS Div layout. For example one of the common paradigms
is that a designer can express that A is next to B or that A is above
B. Concepts like kerning or relative positioning, left alignment,
right alignment, top alignment are all terms we are used to. There's a
design grammar of placement that is abstract - expressing intention
rather than literal coordinates in space. In 3d this design grammar is
obviously much richer given an extra exponent of space. If there were
say 300 kinds of spatial relationships in 2d then in 3d it is probably
300x300 . Are there any examples of 3d design placement rules that
designers use?

One last point - with the hardware limits - I must admit I'm not
totally enthused by the thought of some of the extreme limits that
would be imposed by even supporting HTML at all ( even though I
suppose it is inevitable ).

- anselm

[David Colleen]

Hi Anselm

These are good topics. Luckily, we as a 3D (web3d.org) community have been
working with them for quite some time and the X3D spec has evolved over time
to reflect our needs as we came to understand them. Here is a bit of a
detailed digress on the topic:

Billboards
These are supported in X3D and can be 2D or 3D. They can be text, polygons,
images or even live video. Some viewer makers have added proprietary
extensions to support Flash. They can be fixed or set to rotate to always
face the user. Fixed LOD's can be used or the billboard can be dynamically
scaled. They can also be hyperlinked to other materials or actions. They
also support serious geo-registration.

HUD
HUD's are similar to billboards but are bound to the users viewpoint. They
are useful for control panels and the like that follow a user.

Repulsion
A frequent problem with too many billboards is data clutter. We use scripted
"repulsion" to separate billboards to maintain readability. We also use
distance based visibility to improve readability. Google Earth uses this
concept as well.

Echelon Filtering
Another technique that we use is to group like billboards (or icons) into a
single billboard. This is either user selectable or distance based as you
see in Google Earth.

While I am very much in support of standards development within the AR
community, I believe that 90% of their needs are already available to them
in a mature ISO standard... X3D. I believe that the remaining 10% has
already been developed as proprietary extensions by Fraunhofer (Instant
Reality) and Bit Management (used by Metaio). Good next steps would be to
review these extensions for possible adoption as a standard. I would also
like to do a needs assessment of the full AR community to see what is
missing.

I don't really see a need for FPML as it appears to me to be a reinvention
of the wheel. Please help me to understand if I am missing something.

BTW.. Fraunhofer has had an HTML5 3D viewer (X3DOM) publicly available for
many months. You can try it at http://www.x3dom.org/ and
http://www.web3d.org/x3d/wiki/index.php/X3D_and_HTML5 . Kronos has been
racing to catch up with their version that they call WebGL. Both are great
because they make it much easier for a large community to have a 3D, web
based experience.

Cheers... David Colleen

[Anselm Hook]

Such a good point. I keep forgetting about X3D...

It's weird how I always see X3D as a baroque totally complicated
standard being all kinds of walls... when in fact it is really exactly
what we want.

One important thing is that X3D has a series of profiles that you can
support or not - which makes it far more stripped down than many other
grammars which are "all or nothing".

I suppose that repulsion is a good idea but probably will be not used
heavily when AR environments mature.

In video games there is a concept of priority visibility and priority
audio. Audio and video sequences are tagged by kind. Ambient audio is
flagged as a low priority. Audio channels that can be mixed and
overlaid are flagged that way. Pre-emptive audio alerts such as
"Warning Low Fuel" come in at a higher priority and do not overlap but
may be marked as deferrable.... The same is true with video. A HUD
element queue will deliver things as quickly as it can - with higher
priority items ambulancing through the queue very quickly. There is
contention resolution - low scoring items simply are not shown at all.

For AR the arbitration for jockeying for position in front of the
users eyeballs is especially contentious - but not so entirely
different from a video game. AR objects are coming in from different
vendors. Clearly each vendor has their own interest at heart and wants
to be front and center in this "eyeball commons". They can't
egregiously demand to be the highest priority - that will get them
kicked out - but they will posture to consume as much territory as
possible.

Note that the same is occasionally true in video game development.
Often video game developers fight amongst themselves to have their
game objects be the most important ones. Game developers are often
kids. Sometimes developers don't even work as "teams" rather they work
as a series of individuals fighting for visibility - and they develop
a framework for that battle... there's an internal contention between
developers that affects the game code foundation.

So... I guess all I am saying is that I suggest that on top of X3D we
would build an object visibility priority queue... It defines
absolute priority of a feature, and defines if it is pre-emptive or
can be blended.

> already been developed as proprietary extensions by Fraunhofer (Instant
> Reality) and Bit Management (used by Metaio).

I need to read this.

Hmmm, I applied to Metaio but they never even responded... was always
interested in working there - and especially interested now that I
hear they are actually working on a standards based approach as
opposed to merely hacking something together. This must mean that the
AR ecosystem has a dialogue going on between folks at say Metaio and
Layar etc. Fascinating!!!

> I don't really see a need for FPML as it appears to me to be a reinvention
> of the wheel. Please help me to understand if I am missing something.

Your points about X3D may have invalidated this argument yes.

> BTW.. Fraunhofer has had an HTML5 3D viewer (X3DOM) publicly available for
> many months. You can try it at http://www.x3dom.org/   and
> http://www.web3d.org/x3d/wiki/index.php/X3D_and_HTML5 . Kronos has been
> racing to catch up with their version that they call WebGL. Both are great
> because they make it much easier for a large community to have a 3D, web
> based experience.

Awesome - will look at these.

- anselm

[Anselm Hook]

> It's weird how I always see X3D as a baroque totally complicated
> standard being all kinds of walls... when in fact it is really exactly
> what we want.

I mean "behind" all kinds of walls... such as licensing walls and
bureaucracy etcetera...

Typing too fast.

[Mike McCann]

Hello Anselm,

Can you be more specific about these walls?

Especially, when the FAQ at http://www.web3d.org/about/faq/#process-5
states:

What is the license fee to use X3D?

X3D is an open standard that has no royalties associated with it -  you can use it for free wherever you like.
...

(Of course, to participate fully in its development you do need to be a member of the Web3D Consortium.)

-Mike

[Anselm Hook]

> Can you be more specific about these walls?
>
> Especially, when the FAQ at http://www.web3d.org/about/faq/#process-5
> states:
>
> What is the license fee to use X3D?
>
> X3D is an open standard that has no royalties associated with it -  you can
> use it for free wherever you like.
> ...
>
> (Of course, to participate fully in its development you do need to be a
> member of the Web3D Consortium.)
>
> -Mike

I thought that somebody would respond to that comment.

Actually what I meant is that X3D felt like it has walls - although in
reality it is no different from any other standard - clearly free to
use and the like. Just my own historical biases. I've always been kind
of sensitive to standards like GML which although technically open
were not driven by an open process... and basically sucked. However
X3D comes out of the VRML history - with folks that I know like Donald
Brutzman involved... I watched the VRML history very closely as a game
developer - often hanging out with those folks in fact but I left the
games industry as they all started migrating their thinking over to
X3D.

Anyway... something I wanted to bring to the discussion was to
actually take a look at X3D and provide some comments on that to this
group at some point. It's worth reviewing. The features that I liked
the most are that it is componentized ( unlike VRML ) meaning that one
can cherry pick a set of features to support based on a specific
profile. Also unlike VRML it is more easily extensible. I bet there
are lots of folks defining AR profiles right now in fact.

One thing I didn't realize (in looking at the spec now) is that it has
support for 2D vector graphics and spatialized text... and networking
too ( which I'd be curious to look at more closely ).

Other things I'm curious about is if there is a convention for packing
objects as binary blobs or if it's XML encoded... and I'm also
curious if there are any physics extensions such as
http://www.ode.org/ode-latest-userguide.html ...

[Mike Liebhold]

On 5/5/10 11:10 AM, David Colleen wrote:
> 90% of their needs are already available to them in a mature ISO standard... X3D. I believe that the remaining 10% has already been developed as proprietary extensions by Fraunhofer (Instant
> Reality) and Bit Management (used by Metaio).

Hi David, Can you help find the details of these developments and post
here? Note to Blair: we'd love to learn more about the specs of your
browser too!

> I don't really see a need for FPML as it appears to me to be a reinvention
> of the wheel.

Great question. I guess before we can discuss semantic structures like
screen parameters, and sensing precision, we need to first thrash out
some thoughts in answer to your question by looking at two more criteria:

1. If usability and ease of use and development for AR user experiences
are the top essential criteria for an outer UI shell specification,

2. the second critical requirement is interoperability, for easiest
-extensible- technical and procedeural frameworks for integration of
-ALL- other web served data that might eventually need to be included in
an open AR web experience

There are several candidates, so far, for the outer shell, UI layer of
a standard AR web experience:

- HTML5/canvas/etc. as a default extensible web interface ( starting
with Georgia tech prototype?)
- KML ( including Mobilzy ARML? )
- X3d ( including Frauhofer and Metaio extensions)
- Layar? and other so far proprietary frameworks(?)

I am assuming that data created in any of the above frameworks will
necessarily be subsumed. by the others anyway. so which is, futrhest
along as the best candidate for subsuming all of the other inevitable
AR data on the web:
- text
- images
- graphics
- 3d ( many types)
- streaming

Ok, How to work the tradeoffs, and make reasonable judgements on
Criteria one, and Critiera 2 above?


- Mike

[David Colleen]

Hi Mike

Yesterday, I asked Peter Schickel, from Bitmanagement, and Johannes Bahr,
from Fraunhofer, to join the discussion. I hope that they chime in to give
more detail about what they have built and how to engage with them.

There's also another rock solid commercial viewer from Octaga
(www.octaga.com ) in Norway as well as great open source offerings from
Yumetech in Seattle (http://www.xj3d.org/ ), FreeX3D
(http://freewrl.sourceforge.net/ ), Heilan X3D
(http://www.niallmoody.com/heilan/index.htm ) and SwirlX3D
(http://www.pinecoast.com/p3d/ ).

BR

David Colleen
________

[Blair]

I don't have time to get sucked into a detailed discussion about X3D
right now, but I did want to respond to summarize why we didn't use it
for Polars (our soon-to-be-released open-standards-based AR browser).

Basically, in my view X3D is one of those standards that tries to do
"everything" for 3D on the web, and thus is huge and complicated, does
lots of things poorly, tries to provide mechanisms that sound great
but probably don't do exactly what you want (e.g., the "repulsion"
example someone gave; sounds cute, but this is the kind of thing that
probably never does quite what someone wants for _their_ app, and thus
is more limiting than useful), and so on.

We wanted something lower level and more general, that provided
mechanism and programability instead of policy and complexity. I
don't want something that tries to do 90% of what people need, because
it's that remaining 10% that is where the interesting apps will come
from.

So, we went with a combination of an extended KML (for packaging and
structure) and HTML+COLLODA+Javascript (for a standard and flexible
approach to content) embedded within KML (obviously along with
additional functionality in the browser, exposed to author via a
javascript interface).

One we release the thing and people see what we've done, I'll be
delighted to engage in discussions about the pros and cons of various
approaches!

On May 6, 2:10 pm, Mike McCann <mbarim...@gmail.com> wrote:
> Hello Anselm,
>
> Can you be more specific about these walls?
>

> Especially, when the FAQ at *http://www.web3d.org/about/faq/#process-5
> *states:

>
> What is the license fee to use X3D?
>
> X3D is an open standard that has no royalties associated with it -  you can
> use it for free wherever you like.
> ...
>
> (Of course, to participate fully in its development you do need to be a
> member of the Web3D Consortium.)
>
> -Mike

> *
> *

[David Colleen]

Hi Brian

Welcome to the discussion. I would have really liked to have heard some
facts and science rather than slamming other peoples work with innuendo.

Four years ago, when we began to design our GeoFeeder server, we did in
house testing, at Planet 9, of all of the run time formats. We even tried
Google's flavor of Collada with its' KML wrapper. What we found was that on
average, the KML/Collada files were 40% larger than X3D and of course there
was no support for animation, scripting and the other things that make a
run-time 3D format really useful. Our findings should not have been a
surprise since Sony designed Collada as a storage format. Each vertex was
described in lat long making the files huge.

Many smart people started using Collada because of Google. How did the smart
people at Google decide on Collada? 1. Google Earth actually uses the old
Intrinsic 3D format. Collada support was a much later bolt on to allow user
contributed data after they bought Sketch Up. (SketchUp removed VRML export
after this.) 2. Michael Jones said that the Google Earth team never
evaluated other formats. Remember, before being bought by Google, his team
was partly owned by Sony.... they were part of a family of companies. Many
customers have lamented to me that their KML/Collada/SketchUp city models
would not scale above one city block. Not surprising.

Our team ended up using X3D as our "neutral storage format" as well as our
runtime in our desktop client. We wrote support to ingest Collada and many
other formats that come to us from customers and vendors. Our GeoFeeder
server is in part built using the open source GeoServer. If you would like
our free X3D extensions to this great server, please email me.

BTW... we ended up using Mobile DirectX (md3dm) for our cell phone client
because we started our mobile development work on Win CE and other cell
phone supported formats were very primitive. MD3DM was an "it" format back
then but is no longer supported by Microsoft... ouch.

Also, the OGC (Open Geospatial Consortium), the standards body for mapping
and GIS, uses X3D as their 3D format. It's part of their CityGML spec.

Over the years, I have seen dozens of fashionable 3D formats come and go.
Each was tied to a large corporation and suffered the whims of marketing
departments. Each, in its time, proclaimed its' openness and fooled many.
Please... don't take my word for it though... do the math (and testing)
yourself. Or better yet, let's create a scientific test bed to expose the
good and bad to the harsh, penetrating rays of science.

Are you game?

David Colleen

[David Colleen]

Sorry... I mean Blair.

DC

-----Original Message-----
From: David Colleen [mailto:dcol...@planet9.com]
Sent: Friday, May 07, 2010 7:43 AM
To: 'open...@googlegroups.com'
Subject: RE: [openARweb] Re: FPML :-) Focal Plane Markup Language

Hi Brian

Welcome to the ....

[Blair]

David, please don't get defensive. I was not "slamming other peoples
work with innuendo" and I'm sorry you took it that way.

Perhaps I was a bit harsh, but I don't particularly like X3D; I've
followed it over the years, and know some of the people involved in it
quite well.

Thanks for your detailed comments, I do actually agree with a lot of
them. I'm not tied to KML and COLLADA, for example; they were just
the most suitable things we found. We looked at OGC, and any other
GIS or location-based data standards we could find, but we decided
(for a variety of reasons) to not use most of them. KML + COLLADA +
HTML/Javascript offered the best combination of features, as far as we
could tell. This is just about data size and features, it's also
about languages ways of doing things that are comfortable for current
mobile/web developers.

I should be clear on one thing: I'm far less concerned about "getting
it right and complete and perfect" than I am about "making something
that's the best fit for our target developers." I'm interested in
making tools that work. When I demonstrate our work to real people
(e.g., executives and technical and design folks from Turner and CNN,
new media and interactive designers, heads of researcher at major
phone manufacturers, high up folks in advertising agencies, just to
name the folks I interacted with _last_week_) they are very excited,
because they see it as a good blend of leveraging what they do now and
moving it into a new domain.

My expectation is that we will release this, and once people start
using it, and giving us feedback, it will change, potentially in large
ways.

Now, regarding X3D vs COLLADA. Please not that I was considering X3D
as the top-level data format, not just the 3D format. It might
actually be quite good as a 3D format, and we might very well want to
support it (in addition to a subset of COLLADA, or other formats like
POD). I am not tied to COLLADA; in fact, given way a nightmare it is
trying to find or create a 3D renderer for it, I'd be happy to drop
it. Is there a 3D renderer for X3D, that is open source (useful open
source, not GPL or other infectious licenses), and runs efficiently on
mobiles (high end ones with OpenGLES 2.0)? Because at the end of the
day, I want a format I can use.

As for creating a scientific testbed, it sounds fine, and it would be
great if someone did it, but I can't imagine how this could be
funded; is there any funding for this that you know? I've never had
luck interesting funding agencies or corporations in such things.
Students and staff and equipment aren't free, and such an endeavor
would be expensive. Personally, I've got so many other things going
on, that I can't imagine finding the time to do such a thing. As you
say, there are many formats, and formats can change to suit their
use. So, "proving" anything about existing formats seems like a
fairly uninteresting research contribution (at least to someone like
me, who is happy to use whatever is out there).

Anyway, sorry to offend you.
blair

[ThomasWrobel]

Yes, that clears it up thanks :)

[ThomasWrobel]

> I am imagining that a head-mounted display (or contact lenses - on 10
> years or so) could cache a continuous display that extends 360degrees
> around the frontal field of view, so that if you move your head, the
> data would appear stable, with no network querie for refresh,. This
> implies a spherical projection for rendering, rather than a typicl
> rectangular view on a mobile viewfinder. Agian, this need not be
> probable in the near term, but any markup language should be extensible
> to support this view later on, wiht no need for a completely new markup.
> Similarly it would be ideal to implement a semantic framework that will
> support stereo3D ealrier than later. I can imagine many home
> entertainment use cases for full stereo AR.>  

Ok, yes, we are thinking along the same lines then.

Regarding stereoscopic though; couldn't that just be take from zIndex
formatting? (only in this case the z-index refers to a littoral
horizontal axis towards the viewer, rather then just a layering order)

>> "3.) support for a rich, secure  client-side computing environment
> > with
> > dynamically imported decoders and renders for AR media objects, "
>
> > When you say renderer do you mean for specific media formats, or for
> > rendering the scene as a whole? (ie, the actual 3d-rendering)
> > The first seems quite possible, but the second I'm not so sure is
> > practical.
> > I think the scene-rendering itself would be too tied to the client/
> > platform, especially if the client is displaying multiple sources in
> > the field of view at the same time.
>
> This is a  probably great unsolved problem. HTML5 allows dynamic
> imported decoders using XML, eg. KML ( google earth embedded in a web
> page) digital audio, digital video, and 3D, and ray-traced graphics.  As
> far as I know there's no reason different codecs or renderers can't be
> downloaded, and invoked simultaneously.  But, Honestly I don't know the
> answer for sure, and would appreciate any expert commentary.

I suspect any 2d elements (including video/other plugins) could be
rendered and put to a frame buffer (or something...), which a 3d
renderer could
then take as a texture to display on 3d element as a texture (such as
a plane in mid-air).

I still suspect the renderer would be tied to the browser however.

But, like you, I'm mostly guessing here. Despite being a 3d artist, I
dont know much about the ins/outs of rendering engines.

[ThomasWrobel]

Probably jumping in a little late here;

But wouldn't it be useful for any standard to be able to place any
element relative to any other element?
(So rather then absolute X/Y/Z the co-ordinates are specified relative
to another object?)

Wouldn't this cover basically all the relationships possible on its
own? (granted its not as easy as saying "put this ontop of this" -
but that would merely be a higher level way to express "take this
things co-ordinators + half its height and place this other thing at
those co-ordinates plus half its own height"...assuming its placement
is relative to its centre).

[J. Andrew Rogers]

On Mon, May 10, 2010 at 5:43 AM, ThomasWrobel <dark...@gmail.com> wrote:
> Probably jumping in a little late here;
>
> But wouldn't it be useful for any standard to be able to place any
> element relative to any other element?
> (So rather then absolute X/Y/Z the co-ordinates are specified relative
> to another object?)
>
> Wouldn't this cover basically all the relationships possible on its
> own?  (granted its not as easy as saying "put this ontop of this" -
> but that would merely be a higher level way to express "take this
> things co-ordinators + half its height and place this other thing at
> those co-ordinates plus half its own height"...assuming its placement
> is relative to its centre).

The problem occurs when you are working with data from multiple
sources that generated their positioning data from different frames of
reference. Within the error bounds, there will be significant
variation in placement depending on what data from which you compute a
relative measure. Arbitrating and resolving these apparent
discrepancies is important.

This is essentially the generalized feature registration problem that
has plagued e.g. intelligence agencies. Absent usable absolute frames
of reference, resolving a consistent ground truth takes on the
appearance of a very difficult machine-learning problem. Directly
relevant to AR, this issue routinely shows up in battlefield
simulators based on a "mirror world" anchored to reality.

--
J. Andrew Rogers