OSGi support for JClouds
David Bosschaert
After Adrian Cole presented JClouds to the Cloud Working group at the
OSGi Alliance he mentioned that there is an interest in making JClouds
work in an OSGi environment.
To facilitate this, I have created a working area at github where
anyone who is interested in this effort can collaborate so that we can
do this as a group effort :)
The working area is in my fork of the jclouds project:
https://github.com/bosschaert/jclouds on the OSGi branch.
What I propose to do is this. We create a new demo in the demos/osgi
directory that in functionality is similar to the
'createandlistbuckets' demo but written like you'd want to do this
from an OSGi bundle. I would be happy to start this off.
Then we'll work together changing the code so that the demo work nicely :)
When all is agreed and working someone with push rights to
jclouds/jclouds can then take our work and merge it with the jclouds
master.
Anyone who likes to join the effort, send me your github ID and I'll
make you a collaborator - Adrian and Gustavo are already listed...
Best regards,
David
PS the cloud RFP 133 discussions at the OSGi Alliance are open to
everyone, if you'd like to keep informed of that you can subscribe to
the mailing list here:
https://mail.osgi.org/mailman/listinfo/cloud-workshop
Aled Sage
Excellent to hear that you will be working on making JClouds work well with OSGi.
I previously did a bit of work to get JClouds running in an OSGi Equinox container. To get something working very fast (in terms of development effort) I just put all the JClouds jars in a single OSGi bundle - you can then export the appropriate packages. Here are a few things I found that might be useful when you do the OSGi work for-real:
- Separating core, aws and gogrid into separate bundles failed
due to classloading errors.
A rest.properties file is used to map from provider name (e.g. "gogrid" or "ec2") to the classes that are reflectively loaded (e.g. "ec2.contextbuilder=org.jclouds.aws.ec2.EC2ContextBuilder"). The rest.properties file and the classloading is done from inside jclouds-core, so given the name of a class inside a different bundle it will fail to classload it.
The quickest workaround for me was to put everything in one bundle.
A better alternative would be to include the bundle name in the properties file (e.g. "ec2.bundle=org.jclouds.aws.ec2").
Or on activation of each bundle contribute some named OSGi services for each required type (e.g. the declarative service "<component name="ec2.contextbuilder"><service><provide interface="org.jclouds.rest.RestContextBuilder"/>..." - but not sure if these context builders are stateless or if you need a factory instead).
Or one could try to construct the bundle name from the provider name (e.g. "org.jclouds."+provider).
There are probably many other options out there...
- My bundle classpath ended up looking like the manifest snippet
below (to work with GoGrid and AWS). I confess that some of this
work was done before the Christmas holidays and I don't remember
exactly why things were needed (e.g. is jetty.server really
necessary?)!
Bundle-ClassPath: .,
lib/jclouds-core-1.0-beta-8-jar-with-dependencies.jar,
lib/jclouds-blobstore-1.0-beta-8.jar,
lib/jclouds-compute-1.0-beta-8.jar,
lib/jclouds-jsch-1.0-beta-8.jar,
lib/jclouds-scriptbuilder-1.0-beta-8.jar,
lib/jclouds-aws-1.0-beta-8.jar,
lib/jclouds-gogrid-1.0-beta-8.jar,
lib/jsch-0.1.44.jar,
lib/cglib-nodep-2.2.jar,
lib/objenesis-1.2.jar,
lib/easymock-3.0.jar,
lib/easymockclassextension-3.0.jar,
lib/org.mortbay.jetty.server_6.1.15.v200905151201.jar,
lib/commons-io-1.4.jar
I have not yet investigated which of these are already available as separate OSGi bundles, or created any separate OSGi bundles for them.
- I originally came across the following error:
org.osgi.framework.BundleException: Exception in com.cloudsoftcorp.monterey.jclouds.amazon.Activator.start() of bundle com.cloudsoftcorp.monterey.jclouds.amazon.
...
Caused by: java.lang.IllegalAccessError: tried to access class com.google.gson.InnerClassExclusionStrategy from class com.google.gson.JcloudsGsonBuilder
...
This was because we were classloading gson via another bundle in our application rather than from jclouds-core-with-dependencies.jar. We were (for legacy reasons) using the manifest's "Require-Bundle:" instead of the recommended "Import-Package:". Changing to the latter fixed this.
Cheers, Aled
Adrian Cole
Thanks for owning this issue, David!
Cheers,
-Adrian
--
You received this message because you are subscribed to the Google Groups "jclouds-dev" group.
To post to this group, send email to jclou...@googlegroups.com.
To unsubscribe from this group, send email to jclouds-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/jclouds-dev?hl=en.
David Bosschaert
I put together a starting point for this demo. The demo shows how
JClouds could be used in an OSGi context. The demo currently doesn't
work yet as work needs to be carried out in JClouds to make it work.
This is just a writeup of how I think it should work – so feel free to
comment :)
The demo consists of 3 bundles (see attached .png file)
There is a consumer bundle which is really the important bundle. This
bundle consumes (i.e. uses) the JClouds functionality. The demo uses
the BlobStore API to store and retrieve data.
The consumer has no configuration wrt to which BlobStore is used. The
BlobStore is obtained from the OSGi Service Registry; the consumer
takes whatever BlobStore it can find there.
The actual BlobStore instance is put in the Service Registry by
another bundle. In the demo there are two BlobStore provider bundles.
One that provides a Mock BlobStore (nice for testing) and another one
that provides a BlobStore for Amazon S3. So the BlobStore used by the
consumer bundle depends on which provider bundle is installed.
The Consumer Bundle gets a reference to the BlobStore Service injected
into its main component through OSGi Declarative Services (DS) [1].
It's not essential to use DS, you can also use plain vanilla OSGi
ServiceTrackers but DS makes for a really nice and simple way to use
the OSGi service, the entire code of the demo consumer is [2]
(shamelessly based on the createandlistbuckets demo):
public class Component {
private BlobStore blobStore;
public void activate() {
try {
// Create Container
String containerName = "some-container";
blobStore.createContainerInLocation(null, containerName);
// Add Blob
Blob blob = blobStore.newBlob("test");
blob.setPayload("testdata");
blobStore.putBlob(containerName, blob);
// List Container
for (StorageMetadata resourceMd : blobStore.list()) {
if (resourceMd.getType() == StorageType.CONTAINER
|| resourceMd.getType() == StorageType.FOLDER) {
// Use Map API
Map<String, InputStream> containerMap = blobStore.getContext()
.createInputStreamMap(resourceMd.getName());
System.out.printf(" %s: %s entries%n", resourceMd.getName(),
containerMap.size());
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
public void deactivate() {
// can do cleanup here
}
// OSGi Declarative Services calls this to inject our BlobStore to use
public void setBlobStore(BlobStore bs) {
blobStore = bs;
}
// Used by OSGi Declarative Service to indicate a change in BlobStore
// service
public void unsetBlobStore(BlobStore bs) {
blobStore = null;
}
}
The BlobStore provider bundles currently use the following API to
create a BlobStore instance:
context = new BlobStoreContextFactory().createContext(type,
accesskeyid, secretkey);
While this will probably work, there are potentially different
approaches. One such approach could be to use the OSGi Configuration
Admin Service [3] where a ManagedServiceFactory would be registered by
the JClouds-OSGi integration which could make a BlobStore instance
appear once the appropriate configuration is provided for it. This
would give you a standard configuration-based API to say something
like: I'm creating a configuration instance for an S3 BlobStore with
configuration parameters x, y and z. This could then automatically get
the appropriate BlobStore service registered for you. Effectively this
would mean that the provider bundles as mentioned in the demo are not
needed any more. I think a Configuration Admin-based approach is much
nicer than the current demo setup so I would suggest to ultimately
move towards that...
_The code_
Source code is in the OSGi branch on my fork in github:
http://github.com/bosschaert/jclouds/tree/OSGi/demos/osgi
_To run this demo..._
Well, currently the demo doesn't work yet :) So it's really an attempt
to focus the JClouds-OSGi work in order to get it to work.
Obviously, besides a plain vanilla OSGi Framework you need a JClouds
bundle (or multiple bundles, depending on how that pans out). AFAIK
this doesn't exist yet but thanks Aled for your pointers...
Besides that you need the OSGi Declarative Services Bundle [4] and
possibly the OSGi Compendium interfaces bundle [5]. If we go for a
Config Admin approach we also need that bundle, but let's ignore that
initially...
_Next steps_
1. We need to agree that this is the kind of usage that would be
desired for JClouds in OSGi
2. Then we need to update the JClouds code so that the demo will work.
Best regards,
David
[1] Chapter 112 in the OSGi 4.2 compendium spec:
http://www.osgi.org/Download/Release4V42
[2] https://github.com/bosschaert/jclouds/blob/OSGi/demos/osgi/osgi-demo-consumer/src/main/java/org/jclouds/demo/osgi/consumer/Component.java
[3] Chapter 104 in the OSGi 4.2 compendium spec
[4] http://felix.apache.org/site/downloads.cgi
[5] From http://www.osgi.org/Download/Release4V42 or via
http://repo1.maven.org/maven2/org/osgi/org.osgi.compendium/4.2.0/org.osgi.compendium-4.2.0.jar
Andrew Phillips
> 1. We need to agree that this is the kind of usage that would be
> desired for JClouds in OSGi
> 2. Then we need to update the JClouds code so that the demo will work.
+1 on the approach - when I originally looked at this I had something
similar in mind. One thing I wondered at the time is whether it would
also be possible to have the context retrieved from the registry via a
service lookup with additional arguments or a properties object.
Admittedly, this is pretty close to a "roll your own ConfigService",
but I was basing this on the assumption that the arguments needed for
the context retrieval are essentially static, so there may be a way to
have it injected via DS or Blueprint.
I expect we'll see some comments on the validity of the "static
context args" assumption. And Peter and you will certainly able to
point out whether I'm way off the mark on the suitability of having a
context injected ;-)
From a teaching perspective I think it would also be very useful to
have a vanilla Service Tracker and/or a Blueprint Services demo. Do
you think it'd be worth the effort?
ap
Adrian Cole
This is very good work. I appreciate the pretty pictures, too. I have a couple questions.
1. How do you think we should handle untargetted providers (ex. org.jclouds.api/* modules)?
For example, OpenStack Swift uses the module org.jclouds.api/swift and requires the property swift.endpoint to operate. Others like vcloud, eucalyptus, walrus have the same requirements.
2. How should we handle user contexts? For example, the identity and the provider (and specifically the endpoint of the provider) serve as a primary key of sorts.
3. Any way to cleanly ensure blobstore.getContext().close() is called at the right time?
Cheers, and thanks for pushing this forward.
-Adrian
Gustavo Morozowski
Adrian Cole
Guys,
Can you please sign up for this doodle? Let's have a skype call.
http://www.doodle.com/ad5by2r8zxkubxwc
Cheers,
-Adrian
--
David Bosschaert
availability in doodle, assuming that times are UTC.
Cheers,
David
David Bosschaert
Sorry for the late reply - I was in meeting all week...
On 29 January 2011 18:59, Adrian Cole <adrian...@gmail.com> wrote:
> David,
>
> This is very good work. I appreciate the pretty pictures, too. I have a
> couple questions.
>
> 1. How do you think we should handle untargetted providers (ex.
> org.jclouds.api/* modules)?
> For example, OpenStack Swift uses the module org.jclouds.api/swift and
> requires the property swift.endpoint to operate. Others like vcloud,
> eucalyptus, walrus have the same requirements.
So driving the the BlobStore instances through OSGi Configuration
Admin would actually fit nicely here, especially using
ManagedServiceFactories.
For each BlobStore service that you want to appear you would create a
Configuration object.
This configuration object would have a factoryPID that identifies the
type of blobstore it belongs to. The actual configuration object to
configure the blobstore is actually a map so you can add any module
specific configuration in there.
So as an example you might have the following pids (each with an
example set of configuration values to provide):
org.jclouds.blobstore.mock
org.jclouds.blobstore.s3 {accesskeyid=xxx, secretkey=yyy}
org.jclouds.blobstore.openstack.swift {swift.endpoint=zzz}
A bundle that provides an implementation of the BlobStore service
would register a ManagedServiceFactory with one of these factory PIDs
(e.g. org.jclouds.blobstore.openstack.swift) so when one of those
configuration objects gets created it gets called back so that it can
create the appropriate service. The nice thing about this is that if
the BlobStore instance needs internal classes from a certain
implementation, they don't need to be exposed outside of the provider
bundle. All the external communication (outside of the provider
bundle) goes through the public API.
> 2. How should we handle user contexts? For example, the identity and the
> provider (and specifically the endpoint of the provider) serve as a primary
> key of sorts.
Not 100% sure I fully understand, but I think they can be put on the
configuration object described above. The user creates the
configuration with all its parameters. The desired blobstore service
pops up...
> 3. Any way to cleanly ensure blobstore.getContext().close() is called at the
> right time?
This would also be tied to the lifecycle of the Configuration object.
So the user can delete that if its not needed any more, otherwise the
consuming bundle can automatically delete it when it is stopped.
Cheers,
David
Adrian Cole
-Adrian
David Bosschaert
I've entered my availability for these times in PST as 'David
Bosschaert take 2'.
Cheers.
David
Gustavo Morozowski
Adrian Cole
fyi, all. We're having a skype session tomorrow on osgi. Ping me, if you want in:
1600 CET/0800 PST
-a
--
Adrian Cole
I rebased the fork against master and mildly changed to use guava r08.
I tried running instructions, specifically the "wrap" one.
Can you put in examples into the QUICKSTART for taking these files out of the local maven repository? For example, the syntax seems to be off:
mvn org.apache.felix:maven-bundle-plugin:wrap -DpomFile=/Users/adriancole/.m2/repository//net/oauth/core/oauth/20100527/oauth-20100527.pom -Dfile=/Users/adriancole/.m2/repository//net/oauth/core/oauth/20100527/oauth-20100527.jar
Thanks for getting us closer!
-Adrian
--
Gustavo Morozowski
Andrew Phillips
> worked was the bnd tool [1]:
If you're trying to do this from within Eclipse you could also look at
Neil's bndtools plugin:
https://github.com/njbartlett/bndtools
ap
Andrew Phillips
> property lookup, we can literally pass an instance of a subclass of
> RestContextSpec (ex. S3RestContextSpec)
...in which case the (provider-specific) instance of ContextSpec
actually acts as a "configuration definition" for the context to be
created by the core.
So, in effect, this is the ConfigAdmin [1] service in jclouds. As a
next step one might consider making the core a ManagedServiceFactory
(where ManagedService = Context) and making each provider a
configurator which effectively "preps" the core to create a context
for a specific provider.
One potential benefit is that the "Configuration Admin Service is also
responsible for persisting configuration data" [2], so something like
"context recovery" could work out-of-the-box.
One thing that would differ in this case from most of the examples
that some of the properties in the configuration dictionary may need
to be objects (such as the provider class or suchlike). According to
104.4.2 of the Compendium spec [3], that's not allowed:
"A configuration dictionary contains a set of properties in a
Dictionary object. The value of the property must be the same type as
the set of types specified in the OSGi Core Specification in Figure
3.8 Primary property types"
(and Class is not a primary property type). So some other solution
would have to be found there.
ap
[1]
http://www.osgi.org/javadoc/r4v42/org/osgi/service/cm/ConfigurationAdmin.html
[2]
http://www.dynamicjava.org/articles/osgi-compendium/configuration-admin-service
[3] http://www.osgi.org/download/r4v42/r4.cmpn.pdf
David Bosschaert
something like this was in place the osgi-demo-aws-provider and
osgi-demo-mock-provider modules could be removed from the demo. The
only thing you'd probably want to add for the demo is a simple way to
specify the configuration and pass it to Config Admin.
Cheers,
David
Adrian Cole
Here's the summary of the call today. Please revise these notes, if I'm off.
advantages to osgi
* update the service without updating the client. ex. as long as the api stays the same provided the client is running
* service registry is independent of the code. Ex. clients can use various means to identify and resolve their requirements (blueprint, declaritive services, peaberry, ipojo)
config admin seems to be preferred approach.
In order to expose connection-specific attributes (ex. admin connection vs read-only) markers can be used
*create instances and assign markers; lookup based on markers..
In order to expose both agnostic (BlobStore) and providerSpecific (S3Client) interfaces, register the service multiple times for each desirable interface.
It is useful to be able to request "warm" abstractions (ex. ComputeService with image list primed) to deal with constraints such as 30 second request timeouts in GAE.
It may be desirable to register service wrappers that deal with things like Eventual Consistency. Ex. ConsistencyAwareBlobStore, which blocks until an operation is consistent. In this case, a client who requests strict consistency can get one, even if made out of an EC system.
NEXT ACTION:
* update demo to use at least 2 compatible blobstores (ex. scaleup-storage, aws-s3). allow the user to lookup by iso3166code (ex. DE vs US-CA) and provider-specific api (S3Client)
* clear a path to using this effort as a sandbox for testing OSGi cloud workshop ideas
I can probably get test blobstore accounts. Volunteers appreciated!
Cheers,
-Adrian
David Bosschaert
Great writeup!
> NEXT ACTION:
> * update demo to use at least 2 compatible blobstores (ex.
> scaleup-storage, aws-s3). allow the user to lookup by iso3166code (ex. DE vs
> US-CA) and provider-specific api (S3Client)
Just a couple of thoughts re the above...
What I would do for the demo is at the very least make it work with
mock providers so that anyone can run it easily without the need for
any accounts or anything. E.g. you could create a bunch of instances
of the mock providers with various country codes as service attributes
and then unregister the one the client currently uses to see it
migrate to another suitable at runtime...
Obviously using real clouds makes for a much more compelling demo, but
the mock providers should be easily swapped with some real providers
without much further change to the demo.
I would be happy to help out expanding the demo on the OSGi side and
the mock provider, but I'm a little busy over the coming few weeks (a
release and then a holiday :) So if someone else wants to pick it up
before me that would be great too. In any case, I'll send a note when
I'm available to help to avoid any duplicate work.
Cheers,
David
Gustavo Morozowski
David Bosschaert
Yay! Great stuff, Gustavo!
David
Adrian Cole
+1 Thanks, Gustavo!
Gustavo Morozowski
> NEXT ACTION:
> * update demo to use at least 2 compatible blobstores (ex.
> scaleup-storage, aws-s3). allow the user to lookup by iso3166code (ex. DE vs
> US-CA) and provider-specific api (S3Client)
Adrian Cole
I'd like to do a quick review of the osgi integration gustavo's been working on in preparation for osgi devcon/eclipse con, where we'd have a face to face. Gustavo's progress is outstanding, and I've seen a live demo switching providers with his work (youtube pending). If you can make a online meetup next thurs, please do sign up. I'll be at TSSJS, if you want to meet in person. If you can donate a reasonable screen-share enabled goto meeting, please raise your hand.
http://www.doodle.com/d996yn7dm3ab3nq4
Cheers,
-Adrian
Chris Custine
for this chat. Either way I'll be available for this on Thursday.
--
Chris Custine
My Blog :: http://blog.organicelement.com
Chris Custine
--
Chris Custine
My Blog :: http://blog.organicelement.com
On Fri, Mar 11, 2011 at 11:03, Adrian Cole <adrian...@gmail.com> wrote:
David Bosschaert
Great progress! Very nice video from Gustavo, was that already posted
here? I didn't see it, so here's the link:
http://www.youtube.com/watch?v=SJ7sm3lhKnw
Unfortunately I won't be able to make a meeting on Thursday March 17th
so here are some thoughts over email:
* it would be really nice if the demo would actually perform some
function that involves a blobstore (retrieving/storing data) so that
you can see the dynamic switching use that functionality too. Maybe
the demo already does this and I've overlooked...
* I can't wait to play with this myself :) are there some setup
instructions somewhere?
* I really like the use of Configuration Admin...
Best regards,
David
Adrian Cole
Thanks for chiming in. Sorry that it is a holiday and you cannot make
it. We'll have a follow-up during OSGI con (in fact, we'd love if you
can help arrange!)
For those of you who can make it, we'll start the meeting at 09:00
pacific from TSSJS in Vegas.
Chris, can you setup a meeting for remotes?
Cheers,
-Adrian
Chris Custine
Below you will find a link for an online meeting and a Skype contact
(fuzemeeting) for audio, which works really well for international
attendees.
David, enjoy Saint Patrick's Day :-)
Chris
-----------------------------------------
Meeting Subject: ................ JClouds OSGi Discussion
Meeting Date: ................... Thu Mar 17 2011
Meeting Time: ................... 10:00:00 AM (GMT-07:00) US/Mountain
-----------------------------------------
To join the meeting from your computer or mobile device, click or copy
and paste this URL into your browser:
https://www.fuzemeeting.com/fuze/27321ecd/12591641
To join the audio portion of this meeting, choose your dial in method:
Toll and international dial in: +1 (775) 996-3560
Skype in by dialing contact: fuzemeeting
When prompted enter the room number:
Room : ..........................593293
Having trouble joining this meeting?
Click or copy and paste this URL into your browser to visit the FUZE
Support page:
http://www.fuzemeeting.com/fuzemeeting/support
Thanks for using Fuze Meeting.
David Bosschaert
> Hi, David.
>
> Thanks for chiming in. Sorry that it is a holiday and you cannot make
> it. We'll have a follow-up during OSGI con (in fact, we'd love if you
> can help arrange!)
Maybe we can have a mini face-to-face over lunch at
EclipseCon/OSGiCon? No concall facilities but a nice way to catch up
on everything?
I've created a doodle poll for this here: http://www.doodle.com/t6wtb8ifszrd8qi2
Cheers,
David
Adrian Cole
Ray organized a face-to-face at EclipseCon. There's no registration at EclipseCon to join. If you can come, please do RSVP here!
http://www.meetup.com/jclouds/events/16944454/
Moreover, if you have any topics you'd like to cover, or see covered, please do speak up!
Cheers,
-Adrian
--
Adrian Cole
Some notes before we start
Gustavo will review the jclouds OSGi implementation, which is yet to be merged. It is located here:
https://github.com/bosschaert/jclouds/tree/OSGi
Later, we'll quickly review the RFP for OSGi itself, which I'd like your feelings on. It is attached here:
https://www.osgi.org/bugzilla/show_bug.cgi?id=114
Adrian Cole
If you can, please sign which time you're available. I think Monday would be ideal as we can get together before most of eclipsecon starts
http://www.doodle.com/feqndedaks9yfgey
ciao!
David Bosschaert
I'll be there.
Cheers,
David
Adrian Cole
1. very little changes are required on the jclouds side to get started
* david's fork/branch uses the "fragment bundle" approach which only impacts the pom.xml files
* this is the easiest way to move forward without being intrusive
Gustavo will isolate the pom changes (and a few classloader glitches) in his fork and we'll plan on pulling this into master very soon. Please update issue 42 accordingly [1] To ensure this solution helps one of our users, I'll run it by Cloudsoft early next week, if not sooner.
Gustavo will also push the demo to jclouds-examples [2] where more people can see it
Gustavo will log current status in two wiki pages:
1. hooks needed in the pom file for new providers [3]
2. a page similar to our google appengine wiki [4] showing the current approach for client-side OSGi integration
Andrew and Chris will join Gustavo and David assessing next steps. The least of which include the following:
* investigate a service factory so that each provider doesn't need to copy/paste the example AWSServiceFactory, potentially metatype or blueprint service.
* bundle management: ex. an assembly to help developers find the bundles required to employ jclouds providers
The above should be separate issues.
Thanks again for your help, and hope to see you on the OSGi call, if not in person next week!
-Adrian
[1] http://code.google.com/p/jclouds/issues/detail?id=42
[2] https://github.com/jclouds/jclouds-examples
[3] http://code.google.com/p/jclouds/wiki/DevelopersGuide
[4] http://code.google.com/p/jclouds/wiki/UsingGoogleAppEngine
Adrian Cole
Let's do this over skype.
-Adrian
On Thu, Mar 17, 2011 at 11:07 AM, Adrian Cole <adrian...@gmail.com> wrote:
Adrian Cole
https://www.osgi.org/bugzilla/show_bug.cgi?id=114
The only way for us to effectively use an hour, and also get in changes that can be incorporated (considering we aren't osgi members) is to focus on requirements.
In the RFP, look at section 7, and ignore the rest. There's plenty of room to improve in the rest of the doc, but again, this is about being actionable while we're all on the line.
My goal is for us to add into the above bugzilla thoughts we have on requirements (not architecture or implementation) based on what we've discovered.
Cheers,
-Adrian
Adrian Cole
Adrian Cole
Gustavo Morozowski
Thanks for pushing this.
Any suggestions on how to handle the osgi versions of the non-osgi
library dependencies, like an osgi maven profile or new assembly
project? Or is this out of the project concern, and a wiki page
listing the downloading locations of osgified versions created by
alternative projects should suffice?
- oauth
- bouncycastle
- aopalliance
- javax.inject
- jsr250
- gson
- guava
- jsr305
Gustavo
Andrew Phillips
> library dependencies, like an osgi maven profile or new assembly
> project? Or is this out of the project concern, and a wiki page
> listing the downloading locations of osgified versions created by
> alternative projects should suffice?
>
> - oauth
> - bouncycastle
> - aopalliance
> - javax.inject
> - jsr250
> - gson
> - guava
> - jsr305
When are we actually *using* these bundlified deps? Only in tests and
when creating the distribution? Or elsewhere too?
I think that for the moment, we want to ensure that, if you're not
intending to use jclouds in an OSGi context, you won't have to deal
with any OBRs or bundlified versions of deps.
That would mean not replacing any deps in the "standard" project with
bundlified versions provided by someone like SpringSource, which may
lag behind in versions or whatever.
In terms of creating the distribution, I guess that could mean having
to set up an "osgi" profile for the relevant projects which adds the
bundlified deps and OBRs, and doing clever things in the assembly to
remove the non-OSGi versions.
If you're just including jclouds in your project as a GAV, we would
have to document which <excludes> and additional dependencies you'd
need to ensure you get the OSGi versions.
At some point, one could decide to just say "this is an OSGi project
and it will use OSGi deps only", but looking at the user base I wonder
if that makes sense right now.
ap
Gustavo Morozowski
>> Any suggestions on how to handle the osgi versions of the non-osgi
>> library dependencies, like an osgi maven profile or new assembly
>> project? Or is this out of the project concern, and a wiki page
>> listing the downloading locations of osgified versions created by
>> alternative projects should suffice?
>>
>> - oauth
>> - bouncycastle
>> - aopalliance
>> - javax.inject
>> - jsr250
>> - gson
>> - guava
>> - jsr305
>
> When are we actually *using* these bundlified deps? Only in tests and when
> creating the distribution? Or elsewhere too?
>
Currently we are not using them, it is up to the user to figure out
which deps work and which doesn't and assemble them manually.
> I think that for the moment, we want to ensure that, if you're not intending
> to use jclouds in an OSGi context, you won't have to deal with any OBRs or
> bundlified versions of deps.
> That would mean not replacing any deps in the "standard" project with
> bundlified versions provided by someone like SpringSource, which may lag
> behind in versions or whatever.
>
> In terms of creating the distribution, I guess that could mean having to set
> up an "osgi" profile for the relevant projects which adds the bundlified
> deps and OBRs, and doing clever things in the assembly to remove the
> non-OSGi versions.
>
It would be an improvement to have an osgi profile or sub-project that
excluded the non-osgi deps, and add the ones available in maven
central and sonatype oss repos.
> If you're just including jclouds in your project as a GAV, we would have to
> document which <excludes> and additional dependencies you'd need to ensure
> you get the OSGi versions.
>
> At some point, one could decide to just say "this is an OSGi project and it
> will use OSGi deps only", but looking at the user base I wonder if that
> makes sense right now.
>
> ap
>
Adrian Cole
Ioannis Canellos
I guess either way requires us to first locate or create bundlified deps for our modules, right? Can someone help with this?
I don't have any experience with jclouds (I just started looking around), but I have some experience with OSGi and I guess I could give a hand on finding or creating bundles for jcloud dependencies.
--
Ioannis Canellos
Andrew Phillips
ap
Gustavo Morozowski
list, then we can discuss how to move forward.
Gustavo
On Mon, May 16, 2011 at 3:31 PM, Andrew Phillips <aphi...@qrmedia.com> wrote:
> +1 for magic :-))
>
> ap
>
Adrian Cole
--
Gustavo Morozowski
Ioannis solution using karaf is more elegant, using karaf. Take a look
here - https://github.com/iocanel/karaf-jclouds.
Gustavo
Ioannis Canellos
Ioannis solution using karaf is more elegant, using karaf. Take a look
here - https://github.com/iocanel/karaf-jclouds.
- com.google.guava/guava (I know about guava-osgi but haven't used it).
- net.oauth.core
- org.bouncycastle/bcprov-jdk15
- com.sun.jersey/jersey-core
- com.jamesmurty.utils/java-xmlbuilder
- javax.inject/javax.inject
Adrian Cole
--
David Bosschaert
mentioned (btw Geronimo might have some of these too) then I would do
that. If not I would wrap them in the JClouds build system for the
moment.
The SpringSource Enterprise Bundle Repository is being opened up [1]
and this would be a great place for these bundles, but I don't think
that work is fully up and running yet, but ultimately that might be a
good home for these bundles...
Cheers,
David
[1] http://underlap.blogspot.com/2011/05/opening-up-enterprise-bundle-repository.html
Ioannis Canellos
If they can be consumed through maven from one of the repositories
mentioned (btw Geronimo might have some of these too) then I would do
that.
Carlos Sanchez
>> Ioannis solution using karaf is more elegant, using karaf. Take a look
>> here - https://github.com/iocanel/karaf-jclouds.
>
> I am glad that you liked it. Also note that the "features" leverages OBR,
> which makes it even more appealing.
> However, we still need to find or create OSGi bundles for the following
> artifacts (currently they are osgified on the fly using karaf wrap
> protocol):
>
> com.google.guava/guava (I know about guava-osgi but haven't used it).
> net.oauth.core
> org.bouncycastle/bcprov-jdk15
> com.sun.jersey/jersey-core
> com.jamesmurty.utils/java-xmlbuilder
> javax.inject/javax.inject
>
> I could start creating bundles for those dependencies, the question is where
> would they be best hosted? Should they be added directly as a sub project of
> jclouds? Should we donate them to other projects that provide bundle
> repositories (e.g. ServiceMix, Felix, Spring)?
I had to create OSGI bundles before and dependencies are a PITA.
Ideally the projects should pick up the changes, but in the meantime,
or if they don't, you'll need to host them in the jclouds repo
for example, rebundle com.sun.jersey/jersey-core as
org.jclouds.thirdparty.com.sun.jersey/jersey-core
with all the problems that it will cause to downstream consumers if
they use those dependencies too, because Maven will include both in
the classpath and you'll have to play with exclusions.
Just overriding them in the jclouds repo (com.sun.jersey/jersey-core)
won't work as Maven users may have already downloaded them from
another repo.
Eclipse has rebundled a bunch of dependencies as part of their Orbit project
http://download.eclipse.org/tools/orbit/downloads/drops/R20100519200754/
> --
> Ioannis Canellos
> http://iocanel.blogspot.com
> Apache Karaf Committer & PMC
> Apache ServiceMix Committer
>
Ioannis Canellos
for example, rebundle com.sun.jersey/jersey-core as
org.jclouds.thirdparty.com.sun.jersey/jersey-core
with all the problems that it will cause to downstream consumers if
they use those dependencies too, because Maven will include both in
the classpath and you'll have to play with exclusions.
Compile time dependencies is one thing, runtime dependencies is an other. There is no need to add the "OSGified" dependencies inside your maven poms.
You only need to make sure that they are found on runtime. It all depends on the approach you follow in order to populate your runtime environment.
Gustavo Morozowski
>> Ioannis solution using karaf is more elegant, using karaf. Take a look
>> here - https://github.com/iocanel/karaf-jclouds.
>
> I am glad that you liked it. Also note that the "features" leverages OBR,
> which makes it even more appealing.
> However, we still need to find or create OSGi bundles for the following
> artifacts (currently they are osgified on the fly using karaf wrap
> protocol):
>
> com.google.guava/guava (I know about guava-osgi but haven't used it).
I pick from sonatype oss repo -
https://oss.sonatype.org/content/repositories/releases/com/googlecode/guava-osgi/guava-osgi/
> net.oauth.core
> org.bouncycastle/bcprov-jdk15
> com.sun.jersey/jersey-core
> com.jamesmurty.utils/java-xmlbuilder
> javax.inject/javax.inject
>
> I could start creating bundles for those dependencies, the question is where
> would they be best hosted? Should they be added directly as a sub project of
> jclouds? Should we donate them to other projects that provide bundle
> repositories (e.g. ServiceMix, Felix, Spring)?
> --
> Ioannis Canellos
> http://iocanel.blogspot.com
> Apache Karaf Committer & PMC
> Apache ServiceMix Committer
>
Carlos Sanchez
<gusta...@gmail.com> wrote:
> On Tue, May 17, 2011 at 3:51 AM, Ioannis Canellos <ioc...@gmail.com> wrote:
>>> Ioannis solution using karaf is more elegant, using karaf. Take a look
>>> here - https://github.com/iocanel/karaf-jclouds.
>>
>> I am glad that you liked it. Also note that the "features" leverages OBR,
>> which makes it even more appealing.
>> However, we still need to find or create OSGi bundles for the following
>> artifacts (currently they are osgified on the fly using karaf wrap
>> protocol):
>>
>> com.google.guava/guava (I know about guava-osgi but haven't used it).
>
> I pick from sonatype oss repo -
>
> https://oss.sonatype.org/content/repositories/releases/com/googlecode/guava-osgi/guava-osgi/
you should ask them to enable the sync to central, it's just flip a
switch or open an issue in
https://issues.sonatype.org/browse/OSSRH
Ioannis Canellos
We can use guava-osgi bundle for guava.
I think that jersey-core is already bundled (I will have to verify).
I have already committed javax.inject and bcprov-jdk15 to ServiceMix
and we still are looking a solution for
- com.jamesmurty.utils/java-xmlbuilder
- net.oauth.core
Andrew Phillips
> here, if there is one, and can make an arbitrary decision, if there isn't :)
>
> Guys.. any feelings on this?
+1 on donating/hosting elsewhere. We're trying to pick all our
non-OSGi deps from "standard" sources, so it would be nice if we could
do the same for the OSGi deps.
Who knows, perhaps we're not the only ones using these deps, and
someone else will start maintaining them ;-)
ap
Andrew Phillips
> won't work as Maven users may have already downloaded them from
> another repo.
Yes, let's *please* avoid this scenario with "magic" JARs that have to
come from a certain repo. If we need rebundled deps we need rebundled
deps - it's not nice, but until the projects themselves decide to add
manifests that's just the reality...
ap
Adrian Cole
I suspect oauth folks would be interested in a bundled version, but
not entirely sure.
We use this code in only one place: reading PEM files.
Here are the two classes we use:
import net.oauth.signature.pem.PEMReader;
import net.oauth.signature.pem.PKCS1EncodedKeySpec;
Here's the class we use them in: org.jclouds.crypto.Pems
If someone can get PemsTest working without the oauth dep, we've one
less dependency to manage :p Main issue will be the
PKCS1EncodedKeySpec missing from JCE.
-Adrian
Ioannis Canellos
I also created bundles for java-xmlbuilder and oauth and committed them under service mix bundles. They will probablly get released in the next couple of days.
Till they get released the can be found at: http://svn.apache.org/repos/asf/servicemix/smx4/bundles/trunk/
I will continue working on deploying the rest of jclouds on karaf and see if there are more bundles we need.
Adrian Cole
We all thank you for tracking this down.
Cheers,
-Adrian
Ioannis Canellos
Adrian Cole
Gustavo checked it, and now.. it is in master :)
next steps?
-A
Ioannis Canellos
Adrian Cole
such a donation would be welcome! thanks for putting this together.
Cheers,
-Adrian
Gustavo Morozowski
Gustavo
Ioannis Canellos
Adrian Cole
-Adrian
Ioannis Canellos
Adrian Cole
Thanks for the update, Ioannis. Are you using the provider aws-s3? Can you send a link to the code or instructions needed to reproduce this?
Cheers,
A
> *
> http://iocanel.blogspot.com
>
> Apache Karaf <http://karaf.apache.org/> Committer & PMC
> Apache ServiceMix <http://servicemix.apache.org/> Committer
> *
Andrew Phillips
normally derived from the jclouds.properties file. That is scanned and
all key-value pairs made available to Guice as bindings for @Named
strings...
ap
Quoting Ioannis Canellos <ioc...@gmail.com>:
> *Ioannis Canellos*
> *
> http://iocanel.blogspot.com
>
> Apache Karaf <http://karaf.apache.org/> Committer & PMC
> Apache ServiceMix <http://servicemix.apache.org/> Committer
> *
>
> --
> You received this message because you are subscribed to the Google
> Groups "jclouds-dev" group.
> To post to this group, send email to jclou...@googlegroups.com.
> To unsubscribe from this group, send email to
> jclouds-dev...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/jclouds-dev?hl=en.
>
>
--
Andrew Phillips
qrmedia
Unless expressly stated otherwise, this message is confidential.
Access to this e-mail by anyone else is unauthorised. If you are
not an addressee, any disclosure or copying of the contents of
this e-mail or any action taken (or not taken) in reliance on it
is unauthorised and may be unlawful. If you are not an addressee,
please inform the sender immediately.
This message is confidential and may not be redistributed or
broadcast in whole or part in any form, including but not limited
to any form of internet transmission including email, usenet,
newsgroups, www, irc, icq, etc.
All liability for errors and viruses is disclaimed.
Ioannis Canellos
Ioannis Canellos
Adrian Cole
outcome of the vote!
-Adrian
Ioannis Canellos
Gustavo Morozowski
I want to run jclouds unit tests inside osgi container, any recommendations?
btw karaf war deployer is great, I dropped all the jclouds and deps
there and it worked.
Gustavo
Ioannis Canellos
I want to run jclouds unit tests inside osgi container, any recommendations?
btw karaf war deployer is great, I dropped all the jclouds and deps
there and it worked.
Adrian Cole
I've imported this into https://github.com/jclouds/jclouds-karaf
right now, we get a slight error. I'd recommend addressing this, and
then working to add this to our cloudbees build:
[ERROR] Failed to execute goal
org.apache.karaf.tooling:features-maven-plugin:2.2.0:add-features-to-repo
(add-features-to-repo) on project feature: Cannot find version for:
org.apache.servicemix.bundles/org.apache.servicemix.bundles.netty/ ->
[Help 1]
Cheers!
-Adrian
Ioannis Canellos
right now, we get a slight error. I'd recommend addressing this, and
then working to add this to our cloudbees build:
Adrian Cole
I'm fine with the workaround and agree it is the weirdest bug I've seen. Pretty crazy you figured it out, too! Nice one.
+1 on removing the sample, potentially referring to the examples site instead.
After removing the sample, we have a pom for jclouds-karaf and then one for feature. Does it make sense to flatten them together?
-Adrian
>
> On May 29, 2011 12:23 AM, "Ioannis Canellos" <ioc...@gmail.com> wrote:
> >>
> >> right now, we get a slight error. I'd recommend addressing this, and
> >> then working to add this to our cloudbees build:
> >>
> >
> > That's the weirdest thing I've seen lately, it seems that the license header
> > you've added breaks somehow the features-maven-plugin. Apparently it gets
> > crazy with the @ symbol in the cloudconscious email. I replaced '@' with
> > 'at' and it works. If this solution is unacceptable please let me know so
> > that I can work this around, till this bug gets fixed.
> >
> > The jclouds-karaf project contains a sample called s3-reader. I have already
> > committed a more advanced sample to jclouds-examples, shall I remove the
> > s3-reader from jclouds-karaf (to have all samples in one place?). What do
> > you think?
> >
> > --
> > *Ioannis Canellos*
> > *
> > http://iocanel.blogspot.com
> >
> > Apache Karaf <http://karaf.apache.org/> Committer & PMC
> > Apache ServiceMix <http://servicemix.apache.org/> Committer
>
> > *
Ioannis Canellos
I'm fine with the workaround and agree it is the weirdest bug I've seen. Pretty crazy you figured it out, too! Nice one.
+1 on removing the sample, potentially referring to the examples site instead.
After removing the sample, we have a pom for jclouds-karaf and then one for feature. Does it make sense to flatten them together?
Ioannis Canellos
Ioannis Canellos
Adrian Cole
Both Andrew and I are heading out on holiday, but I can give you
credentials to setup sonatype to release this. Do you mind working
through the process on your own?
Cheers,
-A