Virtualbox implementation

[Andrea]

Since a bunch of months me, Adrian and Mattias are working hard to release the jclouds virtualbox api.

I've finally discovered that vbox CLI used vbox API to offer all the operations - vboxmanage CLI - and probably this will speed up our implementation:

In particular:

    VirtualBoxManager.getSessionObject().getConsole().getGuest().executeProcess(execName, flags, arguments, environment, userName, password, timeoutMS, pid)

will be useful to export ip address instead of using a Statement like the one we have at the moment based on vboxmanage guestproperty enumerate "$VMNAME" --patterns "/VirtualBox/GuestInfo/Net/0/V4/IP" | awk '{ print $4 }' | cut -c 1-14.

and to remove the udev crap from the vm, for debian-based os.

Unfortunately it will not help us using:

    VirtualBoxManager.getSessionObject().getConsole().getGuest().updateGuestAdditions(source, flags)

cause according to the vbox devs this feature is buggy.

Ciao,

Andrea

 

[Andrei Savu]

Nice to see you are making progress on this! Sounds really useful for development. 

-- Andrei Savu / andreisavu.ro

Andrea

 

--
You received this message because you are subscribed to the Google Groups "jclouds-dev" group.
To view this discussion on the web visit https://groups.google.com/d/msg/jclouds-dev/-/jJ8t4mFo4PAJ.
To post to this group, send email to jclou...@googlegroups.com.
To unsubscribe from this group, send email to jclouds-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/jclouds-dev?hl=en.

[Andrea Turli]

We hope that this effort on jclouds can give a boost even to Whirr.

Thanks for the support,
Andrea

[Andrei Savu]

On Wed, Dec 14, 2011 at 5:01 PM, Andrea Turli <andrea...@gmail.com> wrote:

We hope that this effort on jclouds can give a boost even to Whirr.

For sure! Thanks! 

[Adrian Cole]

Thanks for the update, Andrei. and kudos to you and Mattias for
carrying this forward.

Thanks for the info about what works vs doesn't, and bringing
visibility to what you are doing.

comments inline.

On Wed, Dec 14, 2011 at 6:55 AM, Andrea <andrea...@gmail.com> wrote:
> Since a bunch of months me, Adrian and Mattias are working hard to release
> the jclouds virtualbox api.
>
> I've finally discovered that vbox CLI used vbox API to offer all the
> operations - vboxmanage CLI - and probably this will speed up our
> implementation:

particularly where this is modeled in a way that we can switch to API later.

>
> In particular:
>
> VirtualBoxManager.getSessionObject().getConsole().getGuest().executeProcess(execName,
> flags, arguments, environment, userName, password, timeoutMS, pid)
> will be useful to export ip address instead of using a Statement like the
> one we have at the moment based on vboxmanage guestproperty enumerate
> "$VMNAME" --patterns "/VirtualBox/GuestInfo/Net/0/V4/IP" | awk '{ print $4
> }' | cut -c 1-14.
> and to remove the udev crap from the vm, for debian-based os.

The above seems like better modeled as an API call, as metadata such
as IP address info surely must be in a domain object somewhere, right?
In general, I get the idea, though: progress before perfection.

I might consider making a version of RunScriptOnNode that works with
VBoxManager, too.

>
> Unfortunately it will not help us using:
>
> VirtualBoxManager.getSessionObject().getConsole().getGuest().updateGuestAdditions(source,
> flags)
> cause according to the vbox devs this feature is buggy.

So, what's the alternative here? running a script?

[Andrea Turli]

On Wed, Dec 14, 2011 at 5:05 PM, Adrian Cole <adrian...@gmail.com> wrote:
> Thanks for the update, Andrei.  and kudos to you and Mattias for
> carrying this forward.

I'm Andrea but it's ok! :P

> Thanks for the info about what works vs doesn't, and bringing
> visibility to what you are doing.
>
> comments inline.
>
> On Wed, Dec 14, 2011 at 6:55 AM, Andrea <andrea...@gmail.com> wrote:
>> Since a bunch of months me, Adrian and Mattias are working hard to release
>> the jclouds virtualbox api.
>>
>> I've finally discovered that vbox CLI used vbox API to offer all the
>> operations - vboxmanage CLI - and probably this will speed up our
>> implementation:
> particularly where this is modeled in a way that we can switch to API later.

I didn't get this point, sorry. can you rephrase please.

>>
>> In particular:
>>
>> VirtualBoxManager.getSessionObject().getConsole().getGuest().executeProcess(execName,
>> flags, arguments, environment, userName, password, timeoutMS, pid)
>> will be useful to export ip address instead of using a Statement like the
>> one we have at the moment based on vboxmanage guestproperty enumerate
>> "$VMNAME" --patterns "/VirtualBox/GuestInfo/Net/0/V4/IP" | awk '{ print $4
>> }' | cut -c 1-14.
>> and to remove the udev crap from the vm, for debian-based os.
> The above seems like better modeled as an API call, as metadata such
> as IP address info surely must be in a domain object somewhere, right?
>  In general, I get the idea, though: progress before perfection.

You're probably right about the domain object but consider also that
those calls work only if guest additions are installed so maybe they
don't model the domain object.

> I might consider making a version of RunScriptOnNode that works with
> VBoxManager, too.

Probably it is not strictly needed.

>
>>
>> Unfortunately it will not help us using:
>>
>> VirtualBoxManager.getSessionObject().getConsole().getGuest().updateGuestAdditions(source,
>> flags)
>> cause according to the vbox devs this feature is buggy.
> So, what's the alternative here?  running a script?

Yeah in this case a script seems to me the only possibility now. I
submitted a pull request for that a couple of hours ago.

[Adrian Cole]

Inline andrea :)

On Dec 14, 2011 9:19 AM, "Andrea Turli" <andrea...@gmail.com> wrote:
>
> On Wed, Dec 14, 2011 at 5:05 PM, Adrian Cole <adrian...@gmail.com> wrote:
> > Thanks for the update, Andrei.  and kudos to you and Mattias for
> > carrying this forward.
>
> I'm Andrea but it's ok! :P
>
> > Thanks for the info about what works vs doesn't, and bringing
> > visibility to what you are doing.
> >
> > comments inline.
> >
> > On Wed, Dec 14, 2011 at 6:55 AM, Andrea <andrea...@gmail.com> wrote:
> >> Since a bunch of months me, Adrian and Mattias are working hard to release
> >> the jclouds virtualbox api.
> >>
> >> I've finally discovered that vbox CLI used vbox API to offer all the
> >> operations - vboxmanage CLI - and probably this will speed up our
> >> implementation:
> > particularly where this is modeled in a way that we can switch to API later.

When you can run vboxmanage vs api and achieve the same result, it seems like a strategy  pattern where an interface with @ImplementedBy(FavoriteImpl.class) could help?

>
> I didn't get this point, sorry. can you rephrase please.
>
> >>
> >> In particular:
> >>
> >> VirtualBoxManager.getSessionObject().getConsole().getGuest().executeProcess(execName,
> >> flags, arguments, environment, userName, password, timeoutMS, pid)
> >> will be useful to export ip address instead of using a Statement like the
> >> one we have at the moment based on vboxmanage guestproperty enumerate
> >> "$VMNAME" --patterns "/VirtualBox/GuestInfo/Net/0/V4/IP" | awk '{ print $4
> >> }' | cut -c 1-14.
> >> and to remove the udev crap from the vm, for debian-based os.
> > The above seems like better modeled as an API call, as metadata such
> > as IP address info surely must be in a domain object somewhere, right?
> >  In general, I get the idea, though: progress before perfection.
>
> You're probably right about the domain object but consider also that
> those calls work only if guest additions are installed so maybe they
> don't model the domain object.

Sure makes sense

>
> > I might consider making a version of RunScriptOnNode that works with
> > VBoxManager, too.
> Probably it is not strictly needed.

Your call, just a random thought :)

> >
> >>
> >> Unfortunately it will not help us using:
> >>
> >> VirtualBoxManager.getSessionObject().getConsole().getGuest().updateGuestAdditions(source,
> >> flags)
> >> cause according to the vbox devs this feature is buggy.
> > So, what's the alternative here?  running a script?
>
> Yeah in this case a script seems to me the only possibility now. I
> submitted a pull request for that a couple of hours ago.

Will check that out. Thx!

[Andrea]

I'm having a problem with the unregister method in vbox.

If either some other VM refers to some snapshots (linked clone), or you have some leftover diff image...

you need to delete all linked clones first and then the original machine.

I'm trying to see if there is there a way to discover the linked clone of a machine, apparently only if you go over the image list ...

because the behavior is intentional - if one deletes the "original" vm the linked clone should remain operational.

Do you think we need to change the UnregisterMachineIfExistsAndDeleteItsMedia to tackle this case, or we can face this "potential bug" later?

Andrea

[Andrew Phillips]

> If either some other VM refers to some snapshots (linked clone), or you
> have some leftover diff image...
> you need to delete all linked clones first and then the original machine.

If I understand your explanation correctly,
UnregisterMachineIfExistsAndDeleteItsMedia will only work if there are
no linked clones. If this is "correct" according to virtualbox I don't
think it's something that needs to be fixed immediately, especially if
the "cleanup" one needs to do - delete all linked clones - is possible
using some kind of "delete loop" in client code.

In that case, I think it would be OK to document the behaviour and
point to a code sample of how to find all linked clones and delete
them if necessary.

ap

[Andrea Turli]

Thx Andrew.

On Thu, Dec 22, 2011 at 3:05 PM, Andrew Phillips <aphi...@qrmedia.com> wrote:

If either some other VM refers to some snapshots (linked clone), or you
have some leftover diff image...
you need to delete all linked clones first and then the original machine.

If I understand your explanation correctly, UnregisterMachineIfExistsAndDeleteItsMedia will only work if there are no linked clones. If this is "correct" according to virtualbox I don't think it's something that needs to be fixed immediately, especially if the "cleanup" one needs to do - delete all linked clones - is possible using some kind of "delete loop" in client code.

This seems to be the expected behaviour according to vbox community.

In that case, I think it would be OK to document the behaviour and point to a code sample of how to find all linked clones and delete them if necessary.

Actually this is a very good idea: we will add an helper to delete all the linked clones starting from a machine.

Andrea

ap

--
You received this message because you are subscribed to the Google Groups "jclouds-dev" group.

To post to this group, send email to jclou...@googlegroups.com.

To unsubscribe from this group, send email to jclouds-dev+unsubscribe@googlegroups.com.

[Andrea Turli]

I'm trying to implement the functionality to find all linked clones and delete them if necessary.

The idea like in the pipe shell is to compose different small function to solve a bigger problem.

Starting from these 3 operations:

List<IMachine> findClonesOfAMachine(VmSpec)

VmSpec IMachineToVmSpec(IMachine)

UnregisterMachineIfExistsAndDeleteItsMedia(VmSpec)

we can have something like:

findClonesOfAMachine | IMachineToVmSpec | UnregisterMachineIfExistsAndDeleteItsMedia

to delete all the linked clones of a particular machine.

There are some edge cases in this approach, more specifically, this approach is not idempotent, in fact the order of the machines deletion is important: first the linked clones than the "master".

Is it reasonable? What do you think? Any suggestions?

Thanks,

Andrea

[Adrian Cole]

Hi, Andrea.

Good stuff and thanks for pinging jclouds-dev + Thoughts inline.

On Dec 31, 2011 2:43 AM, "Andrea Turli" <andrea...@gmail.com> wrote:
>
> I'm trying to implement the functionality to find all linked clones and delete them if necessary.
>
> The idea like in the pipe shell is to compose different small function to solve a bigger problem.

Warning.. Guava advise to follow :)

> Starting from these 3 operations:
>
> List<IMachine> findClonesOfAMachine(VmSpec)

Iterable<IMachine> clones = Iterables.filter(allMachines, IMachinePredicates.ofSpec(vmspec));

>
> VmSpec IMachineToVmSpec(IMachine)

Iterable<VmSpec> specs = Iterables.transform(clones, IMachines.toVmSpec());
>
> UnregisterMachineIfExistsAndDeleteItsMedia(VmSpec)
Iterable<Boolean> deletedOrDidntExist = Iterables.transform(specs, VmSpecs.unregisterAndDeleteMediaIfExists(vboxmgr));

Note this could be ... Functions.compose(unregisterIfExists,deleteMedia) which is easier to test.

>
> we can have something like:
>
> findClonesOfAMachine | IMachineToVmSpec | UnregisterMachineIfExistsAndDeleteItsMedia
>
> to delete all the linked clones of a particular machine.
>
> There are some edge cases in this approach, more specifically, this approach is not idempotent, in fact the order of the machines deletion is important: first the linked clones than the "master".

You can add an ordering step before delete.

>
> Is it reasonable? What do you think? Any suggestions?

Yep.  Very sensible.

Sounds like you want to emulate a shell pipeline.  Here's some hints:

Grep = Iterables.filter
Sed = Iterables.transform
Sort = Ordering...

Good job on this!  Keep us posted and in the loop.

-A

>
> Thanks,
> Andrea
>
>
> On Thu, Dec 22, 2011 at 3:10 PM, Andrea Turli <andrea...@gmail.com> wrote:
>>
>> Thx Andrew.
>>
>> On Thu, Dec 22, 2011 at 3:05 PM, Andrew Phillips <aphi...@qrmedia.com> wrote:
>>>>
>>>> If either some other VM refers to some snapshots (linked clone), or you
>>>> have some leftover diff image...
>>>> you need to delete all linked clones first and then the original machine.
>>>
>>>
>>> If I understand your explanation correctly, UnregisterMachineIfExistsAndDeleteItsMedia will only work if there are no linked clones. If this is "correct" according to virtualbox I don't think it's something that needs to be fixed immediately, especially if the "cleanup" one needs to do - delete all linked clones - is possible using some kind of "delete loop" in client code.
>>>
>> This seems to be the expected behaviour according to vbox community.
>>
>>> In that case, I think it would be OK to document the behaviour and point to a code sample of how to find all linked clones and delete them if necessary.
>>
>>
>> Actually this is a very good idea: we will add an helper to delete all the linked clones starting from a machine.
>>
>> Andrea
>>>
>>>
>>>
>>> ap
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "jclouds-dev" group.
>>> To post to this group, send email to jclou...@googlegroups.com.

>>> To unsubscribe from this group, send email to jclouds-dev...@googlegroups.com.

>>> For more options, visit this group at http://groups.google.com/group/jclouds-dev?hl=en.
>>>
>>
>

> --
> You received this message because you are subscribed to the Google Groups "jclouds-dev" group.
> To post to this group, send email to jclou...@googlegroups.com.

> To unsubscribe from this group, send email to jclouds-dev...@googlegroups.com.

[Andrea Turli]

Thanks Adrian for the feedback. Inline my comments:

On Sat, Dec 31, 2011 at 5:46 PM, Adrian Cole <adrian...@gmail.com> wrote:

Hi, Andrea.

Good stuff and thanks for pinging jclouds-dev + Thoughts inline.

On Dec 31, 2011 2:43 AM, "Andrea Turli" <andrea...@gmail.com> wrote:
>
> I'm trying to implement the functionality to find all linked clones and delete them if necessary.
>
> The idea like in the pipe shell is to compose different small function to solve a bigger problem.

Warning.. Guava advise to follow :)

> Starting from these 3 operations:
>
> List<IMachine> findClonesOfAMachine(VmSpec)

Iterable<IMachine> clones = Iterables.filter(allMachines, IMachinePredicates.ofSpec(vmspec));

This is a good proposal but at the moment I'm not sure we can have the list of all the machines to be filtered using the vbox api. 

>
> VmSpec IMachineToVmSpec(IMachine)

Iterable<VmSpec> specs = Iterables.transform(clones, IMachines.toVmSpec());

This is implemented and already accepted in jclouds/jclouds. 

>
> UnregisterMachineIfExistsAndDeleteItsMedia(VmSpec)
Iterable<Boolean> deletedOrDidntExist = Iterables.transform(specs, VmSpecs.unregisterAndDeleteMediaIfExists(vboxmgr));

Note this could be ... Functions.compose(unregisterIfExists,deleteMedia) which is easier to test.

I completely agree with this. Good idea.

>
> we can have something like:
>
> findClonesOfAMachine | IMachineToVmSpec | UnregisterMachineIfExistsAndDeleteItsMedia
>
> to delete all the linked clones of a particular machine.
>
> There are some edge cases in this approach, more specifically, this approach is not idempotent, in fact the order of the machines deletion is important: first the linked clones than the "master".

You can add an ordering step before delete.

 

If I have correctly understood you are suggesting to have also a check inside "UnregisterMachineIfExistsAndDeleteItsMedia(machine)" to recursively find & order all the clones of the given machine. 

In fact sometimes you cannot delete a 'master' machine, if it has been linked cloned, because its hd is shared with other clones. In that case it is reasonable to delete all the machines "attached to that master" to finally delete the master.

Maybe we need to improve VmSpec domain class with the list of linked clones. wdyt?

Andrea

[Adrian Cole]

Back at ya

On Jan 3, 2012 3:45 AM, "Andrea Turli" <andrea...@gmail.com> wrote:
>
> Thanks Adrian for the feedback. Inline my comments:
>
> On Sat, Dec 31, 2011 at 5:46 PM, Adrian Cole <adrian...@gmail.com> wrote:
>>
>> Hi, Andrea.
>>
>> Good stuff and thanks for pinging jclouds-dev + Thoughts inline.
>>
>>
>> On Dec 31, 2011 2:43 AM, "Andrea Turli" <andrea...@gmail.com> wrote:
>> >
>> > I'm trying to implement the functionality to find all linked clones and delete them if necessary.
>> >
>> > The idea like in the pipe shell is to compose different small function to solve a bigger problem.
>> Warning.. Guava advise to follow :)
>>
>> > Starting from these 3 operations:
>> >
>> > List<IMachine> findClonesOfAMachine(VmSpec)
>>
>> Iterable<IMachine> clones = Iterables.filter(allMachines, IMachinePredicates.ofSpec(vmspec));
>
> This is a good proposal but at the moment I'm not sure we can have the list of all the machines to be filtered using the vbox api. 

Interesting.  Care to elaborate? what can be listed?  As ComputeService needs to impl listNodes, we need to figure out some strategy for this.

>>
>> >
>> > VmSpec IMachineToVmSpec(IMachine)
>>
>> Iterable<VmSpec> specs = Iterables.transform(clones, IMachines.toVmSpec());
>
> This is implemented and already accepted in jclouds/jclouds. 

Good job on that.

>>
>> >
>> > UnregisterMachineIfExistsAndDeleteItsMedia(VmSpec)
>> Iterable<Boolean> deletedOrDidntExist = Iterables.transform(specs, VmSpecs.unregisterAndDeleteMediaIfExists(vboxmgr));
>>
>> Note this could be ... Functions.compose(unregisterIfExists,deleteMedia) which is easier to test.
>
> I completely agree with this. Good idea.

Cool

>>
>> >
>> > we can have something like:
>> >
>> > findClonesOfAMachine | IMachineToVmSpec | UnregisterMachineIfExistsAndDeleteItsMedia
>> >
>> > to delete all the linked clones of a particular machine.
>> >
>> > There are some edge cases in this approach, more specifically, this approach is not idempotent, in fact the order of the machines deletion is important: first the linked clones than the "master".
>> You can add an ordering step before delete.
>
>  
> If I have correctly understood you are suggesting to have also a check inside "UnregisterMachineIfExistsAndDeleteItsMedia(machine)" to recursively find & order all the clones of the given machine. 
>
> In fact sometimes you cannot delete a 'master' machine, if it has been linked cloned, because its hd is shared with other clones. In that case it is reasonable to delete all the machines "attached to that master" to finally delete the master.
>
> Maybe we need to improve VmSpec domain class with the list of linked clones. wdyt?

By ordering I mean a separate sort step, which orders children first.

VmSpec is probably best used for specs to launch and a name like Vm is better for runtime data like current children, imho. Wdyt?

[Andrea Turli]

On Tue, Jan 3, 2012 at 4:20 PM, Adrian Cole <adrian...@gmail.com> wrote:

Back at ya

On Jan 3, 2012 3:45 AM, "Andrea Turli" <andrea...@gmail.com> wrote:
>
> Thanks Adrian for the feedback. Inline my comments:
>
> On Sat, Dec 31, 2011 at 5:46 PM, Adrian Cole <adrian...@gmail.com> wrote:
>>
>> Hi, Andrea.
>>
>> Good stuff and thanks for pinging jclouds-dev + Thoughts inline.
>>
>>
>> On Dec 31, 2011 2:43 AM, "Andrea Turli" <andrea...@gmail.com> wrote:
>> >
>> > I'm trying to implement the functionality to find all linked clones and delete them if necessary.
>> >
>> > The idea like in the pipe shell is to compose different small function to solve a bigger problem.
>> Warning.. Guava advise to follow :)
>>
>> > Starting from these 3 operations:
>> >
>> > List<IMachine> findClonesOfAMachine(VmSpec)
>>
>> Iterable<IMachine> clones = Iterables.filter(allMachines, IMachinePredicates.ofSpec(vmspec));
>
> This is a good proposal but at the moment I'm not sure we can have the list of all the machines to be filtered using the vbox api. 

Interesting.  Care to elaborate? what can be listed?  As ComputeService needs to impl listNodes, we need to figure out some strategy for this.

Errate Corrige: vbox api offer List<IMachine> manager.getVBox().getMachines(). Problem solved :)

>>
>> >
>> > VmSpec IMachineToVmSpec(IMachine)
>>
>> Iterable<VmSpec> specs = Iterables.transform(clones, IMachines.toVmSpec());
>
> This is implemented and already accepted in jclouds/jclouds. 

Good job on that.

>>
>> >
>> > UnregisterMachineIfExistsAndDeleteItsMedia(VmSpec)
>> Iterable<Boolean> deletedOrDidntExist = Iterables.transform(specs, VmSpecs.unregisterAndDeleteMediaIfExists(vboxmgr));
>>
>> Note this could be ... Functions.compose(unregisterIfExists,deleteMedia) which is easier to test.
>
> I completely agree with this. Good idea.

Cool

In this way we can offer the same functionality offered by vbox api: unregister without delete doesn't make sense to me, in general, but we can imagine some scenarios where it can be useful. 

>>
>> >
>> > we can have something like:
>> >
>> > findClonesOfAMachine | IMachineToVmSpec | UnregisterMachineIfExistsAndDeleteItsMedia
>> >
>> > to delete all the linked clones of a particular machine.
>> >
>> > There are some edge cases in this approach, more specifically, this approach is not idempotent, in fact the order of the machines deletion is important: first the linked clones than the "master".
>> You can add an ordering step before delete.
>
>  
> If I have correctly understood you are suggesting to have also a check inside "UnregisterMachineIfExistsAndDeleteItsMedia(machine)" to recursively find & order all the clones of the given machine. 
>
> In fact sometimes you cannot delete a 'master' machine, if it has been linked cloned, because its hd is shared with other clones. In that case it is reasonable to delete all the machines "attached to that master" to finally delete the master.
>
> Maybe we need to improve VmSpec domain class with the list of linked clones. wdyt?

By ordering I mean a separate sort step, which orders children first.

VmSpec is probably best used for specs to launch and a name like Vm is better for runtime data like current children, imho. Wdyt?

I agree with you in the distinction between vmspec and vm but unfortunately to retrieve the children is not so straightforward. The only strategy I know is to list the vm attached to a particular hard disk. If the hard disk is used by more than 1 machine then you can consider all those machines are clones.

makes sense?

[Adrian Cole]

scroll down :)

sweet

>>
>> >>
>> >> >
>> >> > VmSpec IMachineToVmSpec(IMachine)
>> >>
>> >> Iterable<VmSpec> specs = Iterables.transform(clones,
>> >> IMachines.toVmSpec());
>> >
>> > This is implemented and already accepted in jclouds/jclouds.
>> Good job on that.
>>
>> >>
>> >> >
>> >> > UnregisterMachineIfExistsAndDeleteItsMedia(VmSpec)
>> >> Iterable<Boolean> deletedOrDidntExist = Iterables.transform(specs,
>> >> VmSpecs.unregisterAndDeleteMediaIfExists(vboxmgr));
>> >>
>> >> Note this could be ...
>> >> Functions.compose(unregisterIfExists,deleteMedia) which is easier to test.
>> >
>> > I completely agree with this. Good idea.
>> Cool
>
> In this way we can offer the same functionality offered by vbox api:
> unregister without delete doesn't make sense to me, in general, but we can
> imagine some scenarios where it can be useful.

then stick with unregisterAndDeleteMediaIfExists.

retrieving children is less important than knowing if the VM is a
linked clone or not. if you can somehow tell that it is a linked
clone, then you can order it above those who aren't. Any way to do
this instead?

[Andrea Turli]

This is becoming too long :D

ok 

Are you thinking to a predicate like IsLinkedClone<IMachine? That can be done for sure.

By the way vbox offers 2 different mechanism to cone machine: full and linked. The linked clone can be created faster and occupies less space (diff disk). Cons: the deletion of master and clones is little harder. 

We can even start supporting full clone that it is simpler.

Any preferences?

Andrea

[Adrian Cole]

Hi, Andrea

Thanks for keeping on :)

Right, so if we can tell whether something is a clone or not, seems we can order properly.

Does a full clone have the same delete children first requirement? Or is it completely independent of its source?

I think full clones will be useful to those running i/o heavy or long running vms.  Priority is linked though, imho.

-A

[Andrea Turli]

My comments inline

On Wed, Jan 4, 2012 at 4:08 PM, Adrian Cole <adrian...@gmail.com> wrote:

Hi, Andrea

Thanks for keeping on :)

Right, so if we can tell whether something is a clone or not, seems we can order properly.

Yes, the only thing to understand is if we need to nuke everything always (master + clones) or we need a way to select what to delete. 

Does a full clone have the same delete children first requirement? Or is it completely independent of its source?

The full clone is completely independent, so it's easier its management.

I think full clones will be useful to those running i/o heavy or long running vms.  Priority is linked though, imho.

ok 

[Adrian Cole]

by the sound of it, the ordering step would be needed when deleting
and any satisfy isLinkedClone(true). I hope that if one were to try to
delete a master who had linked clones, we'd get an exception that we
could translate into IllegalStateException

In the above is true, an aggregate delete command who affects
unordered vms (ex. clone1-master1, master1, clone2-master1) could sort
these first (clone1-master1, clone2-master1, master1) so that the
delete will work fine.

If a delete command tries to affect only a master (ex. master1) where
clones still exist, then the user would get a reasonable error back.

-A

[Andrea Turli]

I'm having an interesting conversation with vbox developers about a new feature called IUnattended that the vbox api will support soon.

if you are interested this is the ML discussion: https://forums.virtualbox.org/viewtopic.php?f=10&t=46798&p=214565#p214565

Andrea

[Andrea Turli]

I'm almost finished with the GuestAdditionsInstaller function. I think
we need to change CreateAndInstallVm function cause we can assume that
a vm is ready when the GuestAdditions are installed.

So I'd like to add
private void ensureMachineHasGuestAdditionsInstalled(String vmName) {
machineUtils.lockSessionOnMachineAndApply(vmName,
LockType.Shared, new GuestAdditionsInstaller(context, manager,
executionType));
}

but do you think it makes sense to pass ComputeServiceContext context
to GuestAdditionsInstaller to be able to run inside
GuestAdditionsInstaller something like

context.getComputeService().submitScriptOnNode(vmMetadata.getId(),
statementList,
runAsRoot(true).wrapInInitScript(false));

Thanks in advance

[Adrian Cole]

Hi, Andrea.  q's inline.

On Wed, Feb 8, 2012 at 11:24 PM, Andrea Turli <andrea...@gmail.com> wrote:

I'm almost finished with the GuestAdditionsInstaller function. I think
we need to change CreateAndInstallVm function cause we can assume that
a vm is ready when the GuestAdditions are installed.

So I'd like to add
private void ensureMachineHasGuestAdditionsInstalled(String vmName) {
     machineUtils.lockSessionOnMachineAndApply(vmName,
LockType.Shared, new GuestAdditionsInstaller(context, manager,
executionType));
}

is the lock here needed?  ex. if the command is just a shell script, then maybe just pass the computeservice? or are there vbox api calls involved in guest additions install?

[Andrea Turli]

Hi Adrian,

Right we don't need the lock here, just to run a script.

Thanks,Andrea

[Andrea Turli]

Hi team,

I'm spending some time in vbox to fix LiveTest.

At the moment all tests are ok but those:

Failed tests:
testCreateNodeWithGroupEncodedIntoNameThenStoreCredentials(org.jclouds.virtualbox.compute.VirtualBoxComputeServiceAdapterLiveTest):
imageId(ubuntu-11.04-server-i386) not found
testListLocations(org.jclouds.virtualbox.compute.VirtualBoxComputeServiceAdapterLiveTest):
expected:<false> but was:<true>

Before changing the code, I'd like to better understand:

@Override
public Iterable<Location> listLocations() {
// Not using the adapter to determine locations
return ImmutableSet.<Location>of();
}

contains this comment. What does it mean exactly?

while testCreateNodeWithGroupEncodedIntoNameThenStoreCredentials fails
because of

Template template = context.getComputeService().templateBuilder().build();

How can I improve the default template?

Thanks,
Andrea

[Adrian Cole]

Hi, Andrea.

Thanks for the attempt to summarize, notes below. 

On Fri, Feb 17, 2012 at 4:33 PM, Andrea Turli <andrea...@gmail.com> wrote:

Hi team,

I'm spending some time in vbox to fix LiveTest.

At the moment all tests are ok but those:

Failed tests:
testCreateNodeWithGroupEncodedIntoNameThenStoreCredentials(org.jclouds.virtualbox.compute.VirtualBoxComputeServiceAdapterLiveTest):
imageId(ubuntu-11.04-server-i386) not found
 testListLocations(org.jclouds.virtualbox.compute.VirtualBoxComputeServiceAdapterLiveTest):
expected:<false> but was:<true>

Before changing the code, I'd like to better understand:

  @Override
  public Iterable<Location> listLocations() {
     // Not using the adapter to determine locations
     return ImmutableSet.<Location>of();
  }

virtualbox is not producing locations (ex. datacenters, hosts, etc.), at least not yet.  If there is a test that checks for locations based on the adapter, change or remove it.
 

contains this comment. What does it mean exactly?


while testCreateNodeWithGroupEncodedIntoNameThenStoreCredentials fails
because of

     Template template = context.getComputeService().templateBuilder().build();

How can I improve the default template?

this is not enough information to comment, and a nit is "How can I improve the default template?" is probably not what you are seeking.  Rather you want to figure out why whatever is failing.  You should start by describing what exactly is failing?  is it that no template matches?  If so, what are the available images retrieved with listImages, etc?

There's probabably a stack trace or something related, or some information about the line that failed, etc.  Especially when asking for help offline, you'll want to give us a chance to do so without necessarily running virtualbox itself.  Best way of facilitating your answer is to be somewhat detailed about what you were expecting vs what occurred.

[Andrea Turli]

Hi,

my comments inline.

On Fri, Feb 17, 2012 at 3:42 PM, Adrian Cole <adrian...@gmail.com> wrote:
> Hi, Andrea.
>

> Thanks for the attempt to summarize, notes below.
>
> On Fri, Feb 17, 2012 at 4:33 PM, Andrea Turli <andrea...@gmail.com>
> wrote:
>>
>> Hi team,
>>
>> I'm spending some time in vbox to fix LiveTest.
>>
>> At the moment all tests are ok but those:
>>
>> Failed tests:
>>
>> testCreateNodeWithGroupEncodedIntoNameThenStoreCredentials(org.jclouds.virtualbox.compute.VirtualBoxComputeServiceAdapterLiveTest):
>> imageId(ubuntu-11.04-server-i386) not found
>>
>>  testListLocations(org.jclouds.virtualbox.compute.VirtualBoxComputeServiceAdapterLiveTest):
>> expected:<false> but was:<true>
>>
>> Before changing the code, I'd like to better understand:
>>
>>   @Override
>>   public Iterable<Location> listLocations() {
>>      // Not using the adapter to determine locations
>>      return ImmutableSet.<Location>of();
>>   }
>
> virtualbox is not producing locations (ex. datacenters, hosts, etc.), at
> least not yet.  If there is a test that checks for locations based on the
> adapter, change or remove it.
>

ok, I'll remove it.

>>
>>
>> contains this comment. What does it mean exactly?
>>
>>
>> while testCreateNodeWithGroupEncodedIntoNameThenStoreCredentials fails
>> because of
>>
>>      Template template =
>> context.getComputeService().templateBuilder().build();
>>
>> How can I improve the default template?
>
> this is not enough information to comment, and a nit is "How can I improve
> the default template?" is probably not what you are seeking.  Rather you
> want to figure out why whatever is failing.  You should start by describing
> what exactly is failing?  is it that no template matches?  If so, what are
> the available images retrieved with listImages, etc?
>
> There's probabably a stack trace or something related, or some information
> about the line that failed, etc.  Especially when asking for help offline,
> you'll want to give us a chance to do so without necessarily running
> virtualbox itself.  Best way of facilitating your answer is to be somewhat
> detailed about what you were expecting vs what occurred.
>>
>>

Sorry for the poor description of the problem.

testCreateNodeWithGroupEncodedIntoNameThenStoreCredentials fails with
this stacktrace:

java.util.NoSuchElementException: imageId(ubuntu-11.04-server-i386) not found
at org.jclouds.compute.domain.internal.TemplateBuilderImpl.build(TemplateBuilderImpl.java:636)
at org.jclouds.compute.domain.internal.TemplateBuilderImpl.build(TemplateBuilderImpl.java:622)
at org.jclouds.virtualbox.compute.VirtualBoxComputeServiceAdapterLiveTest.testCreateNodeWithGroupEncodedIntoNameThenStoreCredentials(VirtualBoxComputeServiceAdapterLiveTest.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:691)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:883)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1208)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:753)
at org.testng.TestRunner.run(TestRunner.java:613)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:335)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:330)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:292)
at org.testng.SuiteRunner.run(SuiteRunner.java:241)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1169)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1094)
at org.testng.TestNG.run(TestNG.java:1006)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:107)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:199)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:170)

So the template doesn't match any image. In fact, listImages() return
an empty collection.
We need to ensure that at least the *default master image* is created
before running CreateNodeWithGroupEncodedIntoNameThenStoreCredentials.

Does it make sense?

The other option could be to use the yaml description of the machines
instead of the actual machines in listImages() to support a a lazy
master creation, without touching the template matching mechanism.
This second approach is more generic but not super easy.

Thanks

[Andrea Turli]

Hi guys,

I'd like to organize a quick meeting to summarize the situation for virtualbox.

At the moment the situation is pretty much the following:

- we are able to create a Master machine with GuestAdditions installed
and with the udev network files cleaned in case of ubuntu os.
- we are able to clone from a Master a VM

To do:

- we need to remove the OS iso from the Master
- we need to replace the NAT adapter to Bridged adapter in the Master
to simplify the cloning step

At this regard,
- we need also to review the NetworkSpec class: at the moment contains
reference only to NatAdapter set and I think the general concept is
more generic than that, for example Bridged adapter.
we could decide to have a wrapper class (i.e. NetworkAdapter around
INetworkAdapter)

- VirtualBoxComputeServiceAdapterLiveTest are not passing: basically
because the default template doesn't match any image.

In fact, listImages() return an empty collection. We need to ensure
that at least the *default master image* is created before running
CreateNodeWithGroupEncodedIntoNameThenStoreCredentials.

The other option could be to use the yaml description of the
machines instead of the actual machines in listImages() to support a a
lazy master creation, without touching the template matching
mechanism.

Does it make sense?

I was thinking to organize a meeting to discuss these stuff (and more,
if you want) during this week.

Please use this doodle
http://www.doodle.com/m725xzcvai8ibdx7

Andrea

[Adrian Cole]

I voted.. trying to avoid 9pm friday night :P

Thanks for arranging this.
-A

[Mattias Holmqvist]

Sorry for the late response.

Let's do tonight 20.00!

/M

[Adrian Cole]

here's rough dump of notes.  fun stuff!

https://github.com/jclouds/jclouds/wiki/Virtualbox-Image-Store-Design

-A

[David Alves]

Hi Guys

  I'm working on the vbox implementation and I hit a bit of a snag. 

First let me say what I've done. I'm trying to implement createNodeWithGroupEncodedIntoName:

- made the Adapter return a set of images from a yaml file

- matched a default image with a (meaningless) hardware profile.

- created the master spec from the image (by mapping it to the original yaml image)

- about to create and install the master vm…

The current status is: https://github.com/dralves/jclouds/blob/jclouds-vbox/labs/virtualbox/

The most relevant class is https://github.com/dralves/jclouds/blob/jclouds-vbox/labs/virtualbox/src/main/java/org/jclouds/virtualbox/functions/MasterImages.java

As CreateAndInstallVM and CloneAndRegisterMachineFromIMachineIfNotAlreadyExists are working independently I think I might not be that far from something that works in some way.

The snag I hit is again with guice n00biness. Since CreateAndInstallVm is injected with all kinds of things including the context I cannot make it available by binding in VirtualBoxComputeServiceContextModule because or circular dependencies.

Any suggestions?

Cheers

David Alves

[Mattias Holmqvist]

Hi David!

Nice to see the progress! I also started on a similar path yesterday
but did not get far enough to commit anything. I was planning on
working on vbox tomorrow evening, so maybe we can try and puzzle this
together tomorrow evening (CET) if you have the possibility to be
online?

Cheers, Mattias

[David Alves]

Hi Mattias

Tomorrow evening is good for me after 20 WET or before 17 WET.
meet @ irc right?

Cheers
-david

[Adrian Cole]

I can also help saturday or sunday pacific time.

-A

[David Alves]

For me sunday would also work.

Still with regard to the circular dependency stuff i've searched pretty much everywhere and can't seem to find an example where ComputeServiceContext is injected into a function or predicate as in CreateAndInstallVm. Most everywhere else it seems ComputeServiceContext is not used outside *Builder and *ContextModule which makes me think that it shouldn't be injected at all.

in virtually every case i've seen (in virtualbox) ComputeServiceContext is used to get an instance of ComputeService to submit scripts (e.g GuestAdditionsInstaller). As binding is done by VirtualBoxComputeServiceContextModule *while* ComputeServiceContext is being build it stands to reason that ComputeServiceContext cannot be injected (because its not built yet).

Adrian: any pointers? even a quick clue would be nice.

Cheers
-david

[Adrian Cole]

Hi, David.

I think the trick here is that there's really two computeservices we
should have. The internal one we need to control the vbox host, and
the one we are constructing.

binding the host computeservice explicity, will unravel the circular dependency.

ex.

bind(ComputeService.class).annotatedWith(Host.class).to(fromBYON)

or..

@Provides
@Singleton
@Host (assuming we have or can create an @Qualifier annotation)
protected ComputeService provideHostController(inputs to byon)

Then, in the utilities you need, just bind the qualified ComputeServiceContext.

ex.

@Inject
MyFunction(@Host ComputeService hostController)

...

Here, a diagram could help, and is something we can work on Sunday.

I hope this helps in the mean time.
-A

[David Alves]

Hi Adrian

thanks for the insight, I think I got the gist of it.
I will experiment with that, If I get stuck again we can continue sunday.

thanks
-david

[Adrian Cole]

Fyi.  I got up too early, so I'll be online in a half hour to follow up with stuff including andrea's vbox pull review.

-A

[David Alves]

Hi Guys

I was able to make some progress, I got to start a clone machine, which now fails due to network config problems (everything is done from scratch, e.g, the master is created from scratch if it does not exist etc).

Is there any place where the complete cloning + sshing to a node is done from a master vm ?(I did find cloning but no test as to whether the clone is working, maybe still missing?)

I'm undecided whether do a pull request because more of the "live" tests are failing than before (didn't get around to it). WDYT?

I also missed out or your discussions with regard to networking… What are your thoughts on this? (layout etc) Is this documented anywhere?

Cheers

-david

PS: the link to my current work state is: https://github.com/dralves/jclouds/tree/jclouds-vbox/labs/virtualbox

[David Alves]

Hi 

So I was able to get a bit further, the clone now boots even though no nic is attached. 

So the gist of it is most things work but clone (more precisely clone because multiple clones probably won't work as of now) has no internet access even though I call AttachNATAdapterToMachineIfNotAlreadyExists. Strangely if I clone manually from the master machine all works and a new (eth0) iface appears on the clone machine.

I think I got far enough so I'll submit a pull request even though probably (I didn't have the time to test) other Live tests fail. The default maven profile builds correctly though.

Cheers

-david

[Andrea Turli]

Thank you very much David,

I will look at your code soon.

Andrea

[Adrian Cole]

Hi, David.

Sounds like great progress, and I'm sure Andrea can help sort out the networking glitch.  Good idea to send a pull req.  I'll watch for it.

-A

[David Alves]

Hi Adrian

had already submitted it (https://github.com/jclouds/jclouds/issues/395).

there might be some formatting issues, any pointers to jclouds formatting guidelines (for eclipse particularly).

-david

[David Alves]

Hi Guys

I've been running a lot of tests and vbox is very brittle in its current state.

Even pre master+clone functionality often fails in live tests, but it seems the bigger the number of calls to vbox the higher the chance of failure (simpler tests tend to work more often).

I've just noticed that sometimes tests are failing then if we kill the vbox processes and re-run the tests they work again.

Another thing that happens is some tests if run right after the vbox daemons are started fail with strange messages like: "VirtualBox error: The object functionality is limited" this while calling getName in IMachine.

Do you have any suggestions of what can be done to prevent this erratic behavior? (e.g. better vbox session mgmt and lock mgmt, delays etc…)

Cheers

-david

[Adrian Cole]

Try adding @SingleThreaded to the vbox module for now. and take 2 aspirin :)

[Andrea]

As we are not using particularly vboxwebsrv functionalties, I'm trying to look for another (faster) solution to implement vbox calls from jclouds adapter.

Looking at the vbox documentation I've found that vbox supports SOAP and XPCOM (old article https://blogs.oracle.com/nike/entry/python_api_to_the_virtualbox but just to give you the idea)

I'm trying to run some tests: apparently xpcom is faster than SOAP and it doesn't need a vboxsrv running.

Moreover it uses the same api but it deals less with Session hell, at least till now.

Do you think it's worth to try this solution?

Thanks

[Adrian Cole]

Sounds good.. we just need somebody brave enough to sort out a native
bound adapter :)

-A

> --
> You received this message because you are subscribed to the Google Groups
> "jclouds-dev" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/jclouds-dev/-/PKooR2-kAuEJ.

[Andrea]

Quick update: XPCOM seems fast and simpler than SOAP.

Sounds good.. we just need somebody brave enough to sort out a native
bound adapter :)

So you are suggesting a third way: native bound adapter, correct?

Andrea 

[Adrian Cole]

So, XPCOM would require some bridge, as it is far from ReST. Here's
one thing I found..
https://developer.mozilla.org/en/JavaXPCOM

-A

> --
> You received this message because you are subscribed to the Google Groups
> "jclouds-dev" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/jclouds-dev/-/zOEbn4Qu3iIJ.

[Andrea Turli]

I don't think we need to implement a bridge starting from xpcom. In fact vbox sdk offers already a vboxjxpcom.jar which is a bridge that implements the same vbox api available also via SOAP.

Probably we could have a ReST api layer on top of that vbox bridge.

[Adrian Cole]

heh.. didn't look to see such a jar is available. I'd try using that
in a different package. Ex. a LOT of the code is the same, regardless
of which api is used (ex. image construction, guest additions, etc),
so maybe make a package org.jclouds.virtualbox.xpcom and put things
like a ComputeServiceAdapter that uses xpcom?

wdyt?
-A

[Andrea Turli]

I'd try using that
in a different package.  Ex. a LOT of the code is the same, regardless
of which api is used (ex. image construction, guest additions, etc),
so maybe make a package org.jclouds.virtualbox.xpcom and put things
like a ComputeServiceAdapter that uses xpcom?

wdyt?

I agree with the idea of having a different package: at the moment I'm using a simple eclipse project to test things.

Unfortunately even if xpcom seems faster than SOAP and doesn't require a running vbox web server (good)

the session reliability is still a problem with xpcom bridge as well.

Maybe we could start a brave implementation of a third bridge: native :)

wdyt?

[Adrian Cole]

haha baby steps.

-A

[Andrea Turli]

Hi all,

quick update on jclouds-virtualbox.

The provider is still on labs, but after the last improvements, I've
been able to successfully start up to 30 guests on my laptop without
problems.

I've also tested the jclouds-virtualbox provider with Apache Whirr:

bin/whirr launch-cluster --config recipes/zookeeper.properties
--private-key-file ~/.ssh/id_rsa_whirr --template
osFamily=UBUNTU,osVersionMatches=12.04

where $ cat recipes/zookeeper.properties

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

#
# Deploy an Apache ZooKeeper cluster
#
whirr.cluster-name=zookeeper
whirr.instance-templates=3 zookeeper
whirr.bootstrap-user=toor:password

with default ram size for the guests 1024 Mb, and everything seems to
be in good shape.

I think it is a good result :)

If you are interested in having more details, feel free to ping me.

Thanks everyone for great team work particularly
Adrian Cole, David Alves, Mattias Holmqvist, Andrei Savu, Tony
Batchelli, Andrew Bayer and Patrick Debois.

Ciao,
Andrea

[Adrian Cole]

Nice job.  Merged.

[Andrea Turli]

Hi all,

I've updated the jclouds-virtualbox implementation to the latest release 4.2.0

Here you can find more details about the status
http://code.google.com/p/jclouds/issues/detail?id=384

I'm now waiting for an answer from virtualbox developers about making
available vboxjws-4.2.0.jar available on maven central,
to push this new version.

Cheers,
Andrea

[Andrea Turli]

I've uploaded the vboxjws-4.2.0 to
https://oss.sonatype.org/content/repositories/central-sync/org/virtualbox/vboxjws/4.2.0/

So now jclouds-1.6.0-SNAPSHOT is using the last virtualbox version
which seems much better in terms of session management.

Best,
Andrea

[Adrian Cole]

Thanks for going through the process of getting the jar on the way to
maven, Andrea! Can you summarize what that was in case it needs to be
done again?

-A

[Andrea Turli]

Sure I'll add this information to the jclouds-virtualbox documentation.

Yesterday Ioannis was adding the vritualbox api to the jclouds-karaf
features that are used by jclouds-cli, so that we can use virtualbox
from the cli.
He noticed that vboxjws is LGPL2 licensed which is not compatible with
the ASL license. So he thinks that it is not possible package it
inside the cli itself.

This is a big problem for jclouds and whirr in general, I guess, so we
need to find a solution.

Ioannis was suggesting to ask Oracle to relicense the jar
Technically it is also possible to avoid the usage of vbox web service
and either ssh to host machine and run the vbox command on it
(vagrant/veewee approach) or use java native approach to run the C++
vbox commands.

Thoughts?

Andrea

[Andrea Turli]

From vbox IRC:

> Ioannis was suggesting to ask Oracle to relicense the jar

"The GPL2 license is Oracle policy and no, it is not possible to
relicense it as Apache2."

So apparently this is not an option.

Andrea

[Adrian Cole]

Anybody have a favorite wsdl generator to recommend?

-A

On Oct 8, 2012 1:53 AM, "Andrea Turli" <andrea...@gmail.com> wrote:

Thanks Steve,

this is really interesting I'll give a look later today.

Thanks,
Andrea

On Mon, Oct 8, 2012 at 10:33 AM, Steve Loughran <ste...@hortonworks.com> wrote:

> On 7 October 2012 13:40, Andrea Turli <andrea...@gmail.com> wrote:
>
>> Sure I'll add this information to the jclouds-virtualbox documentation.
>>
>> Yesterday Ioannis was adding the vritualbox api to the jclouds-karaf
>> features that are used by jclouds-cli, so that we can use virtualbox
>> from the cli.
>> He noticed that vboxjws is LGPL2 licensed which is not compatible with
>> the ASL license. So he thinks that it is not possible package it
>> inside the cli itself.
>>
>> This is a big problem for jclouds and whirr in general, I guess, so we
>> need to find a solution.
>>
>> Ioannis was suggesting to ask Oracle to relicense the jar
>> Technically it is also possible to avoid the usage of vbox web service
>> and either ssh to host machine and run the vbox command on it
>> (vagrant/veewee approach) or use java native approach to run the C++
>> vbox commands.
>>
>>

> 1. I've used the command line, but all my proposed "apache chaos" library
> does is kill the VM without warning
>
> https://github.com/hortonworks/availability/blob/master/hmonitor/src/main/groovy/org/apache/chaos/infra/vbox/VBoxManage.groovy
>
> 2. It's a web service API -all it takes is someone to take the service API
> definition and write a new WS client for it. Given the JAX-WS client code
> is built into the current JREs, I'd go with that
>
> someone has already done this in the past
> http://jax-ws-commons.java.net/nonav/virtualbox/
> http://jax-ws-commons.java.net/nonav/virtualbox/apidocs/

[Brian McCallister]

On Sun, Oct 7, 2012 at 6:40 AM, Andrea Turli <andrea...@gmail.com> wrote:
> Sure I'll add this information to the jclouds-virtualbox documentation.
>
> Yesterday Ioannis was adding the vritualbox api to the jclouds-karaf
> features that are used by jclouds-cli, so that we can use virtualbox
> from the cli.
> He noticed that vboxjws is LGPL2 licensed which is not compatible with
> the ASL license. So he thinks that it is not possible package it
> inside the cli itself.
>
> This is a big problem for jclouds and whirr in general, I guess, so we
> need to find a solution.
>
> Ioannis was suggesting to ask Oracle to relicense the jar
> Technically it is also possible to avoid the usage of vbox web service
> and either ssh to host machine and run the vbox command on it
> (vagrant/veewee approach) or use java native approach to run the C++
> vbox commands.
>
> Thoughts?

SSH into the remote machine and run vbox sounds pretty reasonable. The
main alternative would be to write your own wrapper which invokes the
command line tool, and listens on http(s) or something. In that case
you still need to connect to a remote machine, and now you have
another daemon process to monitor, etc.

There are many decent ssh options available, I have found sshj (
https://github.com/shikhar/sshj ) to be pleasant to use if you want
pure ssh, or I have a little convenience wrapper around shelling out
to command line ssh ( https://github.com/brianm/ssh-java-wrapper ) if
you prefer that model.

-Brian

[Andrea Turli]

Thanks Brian for your help!

Even if ssh approach looks reasonable to me as well I've 2 main concerns:
1. the overhead introduced by each ssh call
2. ssh invoking is fine, but retrieving (parsing) the responses can be
a a very hard and painful.

Thoughts?

Andrea

[Andrea Turli]

Thanks Patrick,

the ssh trick can totally help.

Parsing is not the only problem:
without the stub, all the domain classes are gone, so I need to recreate all the pojo classes by hand (IMachine, INetworkAdapter etc)

Thoughts?

Andrea

Il giorno 08/ott/2012 17:46, "Patrick Debois" <patrick...@gmail.com> ha scritto:

ssh overhead can be minimized by using something like ssh contro session, where all your subsequent ssh connection happen over an established ssh tunnel. see ssh controlpath

many of the ssh command like status have a way to have the output in machinereadable output, so parsing is pretty easy.

[Andrea Turli]

Hi all,

after another IRC chat with vbox developers, maybe things are clearing up.

The real license for vobxjws.jar is specified here
https://www.virtualbox.org/browser/vbox/trunk/src/VBox/Main/webservice/MANIFEST.MF.in
in particular http://glassfish.java.net/nonav/public/CDDL+GPL.html

CDDL is extremely liberal and should be compatible with asl (if you
know more please shout)

jclouds-virtualbox can use vboxjws.jar as-is, even if I need to
rebundle and push to maven central the updated pom with the correct
license.

Thanks all for your help,
Andrea

On Mon, Oct 8, 2012 at 7:32 PM, Patrick Debois <patrick...@gmail.com> wrote:
> you might want to check the wsdl integration of virtualbox i once
> experimented with
> see at the bottom of
> http://www.jedi.be/blog/2009/11/17/controlling-virtual-machines-with-an-API/

[Adrian Cole]

nice!

[Andrea Turli]

Finally the new vboxjws jar has been promoted to maven central

I've submitted a new PR with the latest jar
http://search.maven.org/#artifactdetails%7Corg.virtualbox%7Cvboxjws%7C4.2.0-80737%7Cjar

that contains the right license.

Andrea

[Adrian Cole]

Thanks for the tenacity, Andrea!

[David Alves]

Awesome work Andrea!
Looking forward to trying it.

-david

[Andrea Turli]

Hi David,

we need your feedback!

Andrea