Typica as client to Eucalyptus-managed clusters

[Steve Loughran]

Has anyone tried to talk to a Eucalyptus-managed cluster with Typica yet?

I know the OSS product is meant to offer an AWS-compatible API, but
was wondering whether that meant typica could be pointed at another
cluster management endpoint and used to bring up machine images there?

[David]

I don't recall. However, I remember another user wanting to talk to a
substitute service (maybe for SQS) that required a different server
URL. I just haven't gotten that put in, but it might be timely to
address that for both use cases.

David

[dstie...@stainlesscode.com]

I just tried to set up a typica client to Eucalyptus, and it requires
a bit of hacking to get typica to send requests to /services/
Eucalyptus.

Also, I'm now getting an error that has to do with, I think, namespace
compatibility:

Caused by: javax.xml.bind.UnmarshalException: unexpected element
(uri:"http://ec2.amazonaws.com/doc/2008-12-01/",
local:"DescribeImagesResponse"). Expected elements are...

[David Kavanagh]

typica uses the schema provided by the Amazon services, and as such,
uses the namespace (which includes the API date). This is done
specifically because it helps ensure the contract the APIs define.
Are you saying Eucalyptus doesn't use the same API? Does it look the
same, but with a different namespace? In XML land, that isn't the same
because namespaces matter.
I remember the issue with the request URL (it came up recently in
conversation). It is just hard enough to keep up with the updates
Amazon does to the various services!

David

[Daniel Stieglitz]

The request seems to work fine, and I get a DescribeImagesResponse
document back but as you noted the namespaces differ and this is
causing the problem. If I paste the URL into the browser I can view
the response.

From eucalyptus:

<DescribeImagesResponse xmlns="http://ec2.amazonaws.com/doc/
2008-12-01/">

And the error says:

Caused by: javax.xml.bind.UnmarshalException: unexpected element (uri:"http://ec2.amazonaws.com/doc/2008-12-01/

", local:"DescribeImagesResponse"). Expected elements are <{http://ls.amazonaws.com/doc/2008-04-28/
}ActivateDesktopProduct>, (huge list of elements)

This kind of implies that Eucalyptus uses a *newer* API, no?

Dan

[David Kavanagh]

The EC2 namespace that typica supports is the current one from EC2
(http://ec2.amazonaws.com/doc/2009-03-01/)
If you look at the huge list of elements, in there, is probably the
current ec2 namespace. I don't remember which one, but I should think
getting an older version of typica would do the trick. Amazon tends to
support multiple APIs at once, but it sounds like the Eucalypus one
might be a little out of date.

David

[Daniel Stieglitz]

I have the typica source code; would simply getting the correct schema
for amazon and recompiling do the trick? I assume JAXB will slurp up
the schemas.

[David Kavanagh]

Yes, that is true. At one point in time, the ec2.xsd file did support
that version. If you use the older one without using the older code,
things won't compile. The code will be looking for newer features.
It is likely that typica 1.5 or 1.4.1 are what you need. I have tags
created for those, so you could try checking them out of svn.

David

On Thu, Apr 2, 2009 at 7:39 PM, Daniel Stieglitz

[Jérôme BERNARD]

Maybe Typica should start using a catalog?

https://jaxb.dev.java.net/guide/Fixing_broken_references_in_schema.html

We would simply embed both EC2 and Eucalyptus XSD within the JAR and resolve all references to the underlying proper XSDs?

If Eucalypus XSD == EC2 XSD except the namespace version, resolve to the EC2 one instead :-)

Jerome.

2009/4/3 David Kavanagh <dkav...@gmail.com>

--
Jerome Bernard,
Director, EMEA Operations,
Elastic Grid, LLC.
Site: http://www.elastic-grid.com
Blog: http://blog.elastic-grid.com

[Waheed Iqbal]

Hi,

I am using typica to interact with Eucalyptus based cloud. I have
created a small tutorial about modifications in typica to interact
with Eucalyptus( http://waheediqbal.wordpress.com/hack-typica-to-work-with-eucalyptus-14/
), I hope it may help.

Regards
--Waheed

On Apr 3, 6:43 am, David Kavanagh <dkavan...@gmail.com> wrote:
> Yes, that is true. At one point in time, the ec2.xsd file did support
> that version. If you use the older one without using the older code,
> things won't compile. The code will be looking for newer features.
> It is likely that typica 1.5 or 1.4.1 are what you need. I have tags
> created for those, so you could try checking them out of svn.
>
> David
>
> On Thu, Apr 2, 2009 at 7:39 PM, Daniel Stieglitz
>

[Steve Loughran]

On Tue, Apr 7, 2009 at 1:58 PM, Jérôme BERNARD <jerome....@gmail.com> wrote:
> Maybe Typica should start using a catalog?
> https://jaxb.dev.java.net/guide/Fixing_broken_references_in_schema.html
> We would simply embed both EC2 and Eucalyptus XSD within the JAR and resolve
> all references to the underlying proper XSDs?
> If Eucalypus XSD == EC2 XSD except the namespace version, resolve to the EC2
> one instead :-)
>

1. It's hard to blame typica for fundamental design flaws in W3C XML
Schema Language. I think JAXB may be playing up too, being over-fussy
about xmlns values. This is a problem most WS-* stacks end up
encountering.

2. Looking at Waheed's notes, one of the problems is the AWS
authentication protocol using request time, and assuming the client's
clock and TZ settings are always valid
http://waheediqbal.wordpress.com/hack-typica-to-work-with-eucalyptus-14/

I've had problems there with VMWare images talking to AWS classic.
Ideally, the EC2 api should include a method to give the current date
time/TZ at the far end, information that the client can use in future
calls, so it doesn't have to get its own clock in order.

[Jesper]

> Maybe Typica should start using a catalog?https://jaxb.dev.java.net/guide/Fixing_broken_references_in_schema.html

>
> We would simply embed both EC2 and Eucalyptus XSD within the JAR and resolve
> all references to the underlying proper XSDs?
> If Eucalypus XSD == EC2 XSD except the namespace version, resolve to the EC2
> one instead :-)

Another solution could be to simply indicate in some constructor
argument, property or factory which version of the XSD to use in the
client. And then having Typica compile against all versions of the
XSD.

I am looking for a solution where i can use eucalyptus as a test
platform and then shifting the same program to EC2 when ready, by only
changing properties like host-name ports and perhaps API version if
necessary.

What do you think of this? Is someone working on a solution ?

I am more than willing to help with the implementation or testing
whatever you gyus need to get this work done.

Best regards
Jesper

[David Kavanagh]

Since the JAXB generated classes are in their own package, and typica
is coded to those classes, there is a somewhat hard dependency on that
schema.
If eucalypus is trying to support the same apis as EC2, why not have
it use the same WSDL/XSD? In that case, the schema is the same, the
namespace is the same, just need to tweak the end point and we're good
to go. The more I think about it, supporting the APIs really does mean
supporting the same Schema to be fully compliant. I'm sure smarter
people than I have hashed through this issue before.

David

[Jesper]

I looked a bit into JAXB and you are right, it is very hard to get
JAXB to unmarshal if the namespace is a older version, even if the
rest of the xml validates.

The problem is that eucalyptus does conform to a valid AWD namespace,
it is just an older version than the current EC2.

The current EC2 is 2009-04-04
Typica is currently on 2009-03-01
And eucalyptus is 2008-05-05

Eucalyptus will eventually update the schema version, when they
implement the features, but there is no way of knowing if amazon and
eucalyptus will have the same version at any given time.

I also looked a bit into how amazon are changing the interface, and i
havent seen a breaking change for xml backwards compatibility. Also it
is sane for Amazon to support older version, but Eucalyptus will have
a hard time supporting API versions that they have not implemented
yet.

I dont know what the right solution is, perhaps to be able to specify
the version to use as i asked erlier, but i can understand that this
will be a major change, because you will have to compile the XSD's
into different packages, and then create mappers for each type for
each version, to the typica types.

Meanwhile i will be running on a solution where i will intercept the
response XML and modify the namespace if the version is older than the
one specified by the Version header in the request. It is no a very
clean solution but it will enable me to run seamlessly on Eucalyptus
and EC2.

Best regards
Jesper

On May 18, 9:21 pm, David Kavanagh <dkavan...@gmail.com> wrote:
> Since the JAXB generated classes are in their own package, and typica
> is coded to those classes, there is a somewhat hard dependency on that
> schema.
> If eucalypus is trying to support the same apis as EC2, why not have
> it use the same WSDL/XSD? In that case, the schema is the same, the
> namespace is the same, just need to tweak the end point and we're good
> to go. The more I think about it, supporting the APIs really does mean
> supporting the same Schema to be fully compliant. I'm sure smarter
> people than I have hashed through this issue before.
>
> David
>

[David Kavanagh]

Yea, It's hard enough to keep typica update with all the amazon
changes, I imagine keeping eucalyptus updated also is a bit harder. I
guess my suggestion for now is to use an older version of typica that
supports the same API version that eucalyptus supports. Beyond that, a
change for the service endpoint URL is the only other sticking point,
right?
BTW, I've created tags for the past releases (though I did miss one or two).

David

[David]

I just found out that project I'll be involved with this summer will
involve some interoperability between EC2 and a Eucalyptus (local)
cloud. So, the good news is that I'll get some up close and personal
time with this issue!

David

On May 19, 8:53 am, David Kavanagh <dkavan...@gmail.com> wrote:
> Yea, It's hard enough to keep typica update with all the amazon
> changes, I imagine keeping eucalyptus updated also is a bit harder. I
> guess my suggestion for now is to use an older version of typica that
> supports the same API version that eucalyptus supports. Beyond that, a
> change for the service endpoint URL is the only other sticking point,
> right?
> BTW, I've created tags for the past releases (though I did miss one or two).
>
> David
>

[Jérôme BERNARD]

On Thu, May 28, 2009 at 20:20, David <dkav...@gmail.com> wrote:

I just found out that project I'll be involved with this summer will
involve some interoperability between EC2 and a Eucalyptus (local)
cloud. So, the good news is that I'll get some up close and personal
time with this issue!

Actually, also gonna be involved with this in about 2 or 3 weeks as I need to get Elastic grid working with Eucalyptus too ;-)



Jerome.
 

[David]

when I get out from behind this proxy, I'll commit a change that
allows the "/services/Eucalyptus" part of the URL to be defined. There
is another issue (aside from the XML namespace one) that is about the
timezone for the local cluster. The request time needs to match that?
I saw how Waheed (in this thread) needed to change that code, but I've
made a request to the local cluster and arrive a the namespace issue,
so I think it doesn't affect me. how about others? Should we allow
changing of the request timestamp time zone, or is that more of a
"local cluster" configuration detail?

David

On May 28, 2:52 pm, Jérôme BERNARD <jerome.bern...@gmail.com> wrote:

[David]

cross referencing a post in the Eucalyptus forum. Things are looking
up!
http://forum.eucalyptus.com/forum/viewtopic.php?f=4&t=792&p=3379#p3379

[David]

So, assuming eucalyptus 1.5.2 is the golden goose, Here's the
initialization code you'd use for Jec2.

Jec2 ec2 = new Jec2(props.getProperty("aws.accessId"),
props.getProperty("aws.secretKey"), false, "your-host", 8773);
ec2.setResourcePrefix("/services/Eucalyptus");
ec2.setSignatureVersion(1);

Not certain if SSL works properly, but it didn't with my cluster.

David

On Jul 14, 2:09 pm, David <dkavan...@gmail.com> wrote:
> cross referencing a post in the Eucalyptus forum. Things are looking

> up!http://forum.eucalyptus.com/forum/viewtopic.php?f=4&t=792&p=3379#p3379

[David]

I've been testing against the 1.5.2 release and it looks pretty good.
The error responses come in a SOAP envelope (!?) but I've added some
code in to parse the message out (until they fix that in 1.6). Also,
sig version 2 should work.. will test that as well.