Re: Exposing/Extending an OpenStack specific command

[Adrian Cole]

Probably more a jclouds-dev question.

Is the command only for servers? Do you have a doc link?

If only for servers, update classes: ServerApi ServerAsyncApi ServerApiExpectTest ServerApiLiveTest

Cheers,
-A

On Oct 15, 2012 6:32 AM, "Leander" <lean...@gmail.com> wrote:

Hello all,

I'm currently using version 1.5.2 and was wondering how I could expose (or extend in case it does not exist) the diagnostics command in the OpenStack driver: 

[API_ENDPOINT]/v2/[TENANT]/servers/[SERVER_ID]/diagnostics? 

What would be the best place to implement this, in case the command implemented?

Regards,

Leander

--
You received this message because you are subscribed to the Google Groups "jclouds" group.
To view this discussion on the web visit https://groups.google.com/d/msg/jclouds/-/V9C3Pshi1eMJ.
To post to this group, send email to jcl...@googlegroups.com.
To unsubscribe from this group, send email to jclouds+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/jclouds?hl=en.

[Adrian Cole]

Cool.  While you are on it, you mind pasting what the debug output is?  I'm curious.

-A

On Oct 16, 2012 1:26 AM, "Leander Bessa Beernaert" <lean...@gmail.com> wrote:

Sorry about that, shall direct it to the appropriate list next time.  Thanks for the suggestions, I shall look into to those. There is no doc link, as far as I know of, I've retrieved the link by using the python command line tool with the --debug option for the command diagnostics. This command should work with OpenStack Folsom using the Xen API and libvirt ( more hypervisors might support this, although I'm not sure).

[Leander]

Since i'm using Essex at the moment, the command doesn't work. The only output it produces is below. If you want to check the output of this command, you can find a sample of Xen and libvirt in this commit log https://github.com/openstack/nova/commit/ad54ed53cf6a475ad0f8042f8b95454a8c0b35a4

REQ: curl -i http://10.0.107.2:8774/v2/4ebf76425efa4d01bc54a182185444a6/servers/9313c838-e004-4993-96c0-e75c0e3efb77/diagnostics -X GET -H "X-Auth-Project-Id: project" -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: 5e9ef4cb0d1b43c8b832e2032b7eaf78"

send: u'GET /v2/4ebf76425efa4d01bc54a182185444a6/servers/9313c838-e004-4993-96c0-e75c0e3efb77/diagnostics HTTP/1.1\r\nHost: 10.0.107.2:8774\r\nx-auth-project-id: project\r\nx-auth-token: 5e9ef4cb0d1b43c8b832e2032b7eaf78\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n'

reply: 'HTTP/1.1 500 Internal Server Error\r\n'

header: Content-Length: 128

header: Content-Type: application/json; charset=UTF-8

header: X-Compute-Request-Id: req-097428e6-356d-46da-b497-3919caf08762

header: Date: Tue, 16 Oct 2012 15:34:17 GMT

RESP:{'date': 'Tue, 16 Oct 2012 15:34:17 GMT', 'status': '500', 'content-length': '128', 'content-type': 'application/json; charset=UTF-8', 'x-compute-request-id': 'req-097428e6-356d-46da-b497-3919caf08762'} {"computeFault": {"message": "The server has either erred or is incapable of performing the requested operation.", "code": 500}}

DEBUG (shell:534) The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-097428e6-356d-46da-b497-3919caf08762)

Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/novaclient/shell.py", line 531, in main

    OpenStackComputeShell().main(sys.argv[1:])

  File "/usr/local/lib/python2.7/dist-packages/novaclient/shell.py", line 467, in main

    args.func(self.cs, args)

  File "/usr/local/lib/python2.7/dist-packages/novaclient/v1_1/shell.py", line 819, in do_diagnostics

    utils.print_dict(cs.servers.diagnostics(server)[1])

  File "/usr/local/lib/python2.7/dist-packages/novaclient/v1_1/servers.py", line 425, in diagnostics

    base.getid(server))

  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 199, in get

    return self._cs_request(url, 'GET', **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 186, in _cs_request

    **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 168, in _time_request

    resp, body = self.request(url, method, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 162, in request

    raise exceptions.from_response(resp, body)

ClientException: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-097428e6-356d-46da-b497-3919caf08762)

ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-097428e6-356d-46da-b497-3919caf08762)

[Leander Bessa Beernaert]

I've found the ServerAPI file under ./jclouds/apis/openstack-nova/src/main/java/org/jclouds/openstack/nova/v2_0/features/ServerApi.java. Any pointers on how to compile changes using the eclipse setup and using JClouds REST API?

--
You received this message because you are subscribed to the Google Groups "jclouds-dev" group.
To view this discussion on the web visit https://groups.google.com/d/msg/jclouds-dev/-/ICLsBF5WmnoJ.
To post to this group, send email to jclou...@googlegroups.com.
To unsubscribe from this group, send email to jclouds-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/jclouds-dev?hl=en.

[Richard Downer]

(Replying just to jclouds-dev)

On 18 October 2012 10:45, Leander Bessa Beernaert <lean...@gmail.com> wrote:

I've found the ServerAPI file under ./jclouds/apis/openstack-nova/src/main/java/org/jclouds/openstack/nova/v2_0/features/ServerApi.java. Any pointers on how to compile changes using the eclipse setup and using JClouds REST API?

Firstly, there's actually two files you need to change - ServerApi.java and ServerAsyncApi.java. These files are very similar - both should define the same set of methods with the same parameters. The method return types differ slightly in that every return type in the AsyncApi is wrapped with a ListenableFuture<T>. So if the Api class defines a method returning Server, the equivalent method in AsyncApi will return ListenableFuture<Server>; void maps to ListenableFuture<Void>.

(Implementation detail: at runtime jclouds creates a proxy for the synchronous Api class. The proxy will, for every method, simply call the equivalent AsyncApi method, and then immediately wait on the completion of the ListenableFuture.)

The AsyncApi class also contains lots of annotations - these are crucial as jclouds uses these to determine how to form the request and handle the response. Within a single cloud API these are generally very similar so you can probably copy-and-paste annotations from a similar method (say, get server details) and then modify the annotations for the operation you are working on.

In answer to your actual question: everybody's workflow is slightly different and I don't use Eclipse so can't tell you the best way to use that. However what will probably work is if you build from the command line using "mvn clean install" in the apis/openstack-nova folder. This will build the code and install it in your local maven repository. You can then build your application with maven and it will use the jclouds version in your local repository that you have just built.

Cheers

Richard.

-- 

Richard Downer • Lead Engineer • Cloudsoft Corporation • http://www.cloudsoftcorp.com

Skype richardcloudsoft • Twitter @FrontierTown

[Leander Bessa Beernaert]

Thanks, for the info. It turned out to be surprisingly easy :D. 

However, I'm trying to get around something. This OpenStack command may not be supported on certain hypervisors. How can i make it so that it gracefully handle the http error 500. I currently have the following in ServerAsyncApi.java:

@GET

@Path("/servers/{id}/diagnostics")

@Consumes(MediaType.APPLICATION_JSON)

@ExceptionParser(MapHttp4xxCodesToExceptions.class)

ListenableFuture<String> getDiagnostics(@PathParam("id") String id); 

Since it isn't supported on my current OpenStack Install, it produces the error 500 and the output below is slightly different . Is this expected?

[i/o thread 0] ERROR org.jclouds.http.handlers.BackoffLimitedRetryHandler - Cannot retry after server error, command has exceeded retry limit 5: [method=ServerAsyncApi.getDiagnostics, request=GET http://10.0.107.2:8774/v2/4ebf76425efa4d01bc54a182185444a6/servers/52513b64-42a3-4a68-99b7-eb6d3bc82085/diagnostics HTTP/1.1]

org.jclouds.http.HttpResponseException: command: GET http://10.0.107.2:8774/v2/4ebf76425efa4d01bc54a182185444a6/servers/52513b64-42a3-4a68-99b7-eb6d3bc82085/diagnostics HTTP/1.1 failed with response: HTTP/1.1 500 Internal Server Error; content: [{"computeFault": {"message": "The server has either erred or is incapable of performing the requested operation.", "code": 500}}]

at org.jclouds.openstack.nova.v2_0.handlers.NovaErrorHandler.handleError(NovaErrorHandler.java:48)

at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:71)

at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.shouldContinue(BaseHttpCommandExecutorService.java:197)

at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:167)

at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:135)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

at java.util.concurrent.FutureTask.run(FutureTask.java:138)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:680)

at org.jclouds.concurrent.config.DescribingExecutorService.submit(DescribingExecutorService.java:89)

at org.jclouds.http.internal.BaseHttpCommandExecutorService.submit(BaseHttpCommandExecutorService.java:132)

at org.jclouds.http.TransformingHttpCommandExecutorServiceImpl.submit(TransformingHttpCommandExecutorServiceImpl.java:54)

at org.jclouds.http.TransformingHttpCommandImpl.execute(TransformingHttpCommandImpl.java:73)

at org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFutureForHttpRequestMappedToMethodAndArgs(AsyncRestClientProxy.java:255)

at org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:149)

at $Proxy81.getDiagnostics(Unknown Source)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:170)

at $Proxy82.getDiagnostics(Unknown Source)

at gsd.JCloudsOpenStack.listImages(JCloudsOpenStack.java:72)

at gsd.JCloudsOpenStack.main(JCloudsOpenStack.java:34)

at org.jclouds.concurrent.config.DescribingExecutorService.submit(DescribingExecutorService.java:89)

at org.jclouds.http.internal.BaseHttpCommandExecutorService.submit(BaseHttpCommandExecutorService.java:132)

at org.jclouds.http.TransformingHttpCommandExecutorServiceImpl.submit(TransformingHttpCommandExecutorServiceImpl.java:54)

at org.jclouds.http.TransformingHttpCommandImpl.execute(TransformingHttpCommandImpl.java:73)

at org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFutureForHttpRequestMappedToMethodAndArgs(AsyncRestClientProxy.java:255)

at org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:149)

at $Proxy81.getDiagnostics(Unknown Source)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:170)

at $Proxy82.getDiagnostics(Unknown Source)

at gsd.JCloudsOpenStack.listImages(JCloudsOpenStack.java:72)

at gsd.JCloudsOpenStack.main(JCloudsOpenStack.java:34)

[Richard Downer]

Hi,

I think that something for @ExceptionParser is what you need. Create a class that implements Function<Exception, Object> (my guess is that org.jclouds.openstack.nova.v2_0.functions is the right package for it to belong), and change the @ExceptionParser annotation to refer to your new class.

Richard.

[Leander Bessa Beernaert]

I've been sidetracked for a while, but now i'm back to resolving this issue.

There is another problem i'm currently facing. I've forked JClouds on GitHub so that I can later push these changes back to it. However, I can seem to compile it or install it. 

It hangs at this error:

Tests run: 753, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 55.663 sec <<< FAILURE!

testEventBusIsSingleton(org.jclouds.events.config.EventBusModuleTest)  Time elapsed: 0.001 sec  <<< FAILURE!

java.lang.AssertionError: expected [true] but found [false]

at org.testng.Assert.fail(Assert.java:94)

at org.testng.Assert.failNotEquals(Assert.java:494)

at org.testng.Assert.assertTrue(Assert.java:42)

at org.testng.Assert.assertTrue(Assert.java:52)

at org.jclouds.events.config.EventBusModuleTest.testEventBusIsSingleton(EventBusModuleTest.java:77)

Also, since i'm a bit inexperienced with Maven, how can tell a test project to use my version of JClouds instead of the stable one I'm currently using?

Regards,

Leander

--
You received this message because you are subscribed to the Google Groups "jclouds-dev" group.

[Leander Bessa Beernaert]

Small update, the tests work with mvn clean test but fail when i try mvn install.

[Leander]

I've managed to install it while disable test execution. Now I'm trying to get the diagnostics call to work. 

So far I have the following:

in org.jclouds.openstack.nova.v2_0.features.ServerAsyncApi:

@GET

   @Path("/servers/{id}/diagnostics")

   @Produces(MediaType.APPLICATION_JSON)

   @ResponseParser(DiagnosticsOrNull.class)

   @ExceptionParser(ReturnNullOnNotFoundOr404.class)      

   ListenableFuture<String> getDiagnostics(@PathParam("id") String id);

In org.jclouds.openstack.nova.v2_0.features.ServerApi:

String getDiagnostics(String id);

And org.jclouds.openstack.nova.v2_0.functions.internal.DiagnosticsOrNull:

@Singleton

public class DiagnosticsOrNull implements Function<HttpResponse, String> {

    @Override

    public String apply(HttpResponse f) {

        System.out.println("DiagnosticsOrNull:" +f);

        if (f.getStatusCode() != 500) {

            return f.getMessage();

        } else {

            return null;

        }

    }

}

When i try the code below, ret is always null and the above print does not occur. Any ideas of what I might be doing wrong?

RestContext<NovaApi, NovaAsyncApi> nova =  context.unwrap();

String ret =nova.getApi().getServerApiForZone("nova").getDiagnostics(id);

Regards,

Leaner

To unsubscribe from this group, send email to jclouds-dev+unsubscribe@googlegroups.com.

[Adrian Cole]

Hi, Leander.

Can you report back mvn -version?

[Adrian Cole]

Hi, there.

Can you send us the output of 'jclouds.headers' and 'jclouds.wire' log categories set to debug?

This will help us understand why you always get null.

-A

To view this discussion on the web visit https://groups.google.com/d/msg/jclouds-dev/-/acBEtN8Rah8J.

To post to this group, send email to jclou...@googlegroups.com.

To unsubscribe from this group, send email to jclouds-dev...@googlegroups.com.

[Leander Bessa Beernaert]

mvn version:

Apache Maven 3.0.3 (r1075438; 2011-02-28 17:31:09+0000)

Maven home: /usr/share/maven

Java version: 1.7.0_07, vendor: Oracle Corporation

Java home: /Library/Java/JavaVirtualMachines/jdk1.7.0_07.jdk/Contents/Home/jre

Default locale: en_US, platform encoding: UTF-8

OS name: "mac os x", version: "10.7.5", arch: "x86_64", family: "mac"

Where can I find those files?

[Adrian Cole]

Hi, Leander.

Switch your java home to 1.6 and see if you still get build fail.

When you run live test, you should see the following log:

target/test-data/jclouds-wire.log

You only need to send the relevant section with diagnostics in it.

Cheers,
-A
p.s. I'm not on freenode #jclouds now, but others might be

[Leander Bessa Beernaert]

This is what happend after trying mvn install with jdk 1.6 :/

ests run: 341, Failures: 4, Errors: 0, Skipped: 0, Time elapsed: 32.615 sec <<< FAILURE!

testGetExtensionByAliasWhenResponseIs2xx(org.jclouds.openstack.nova.v2_0.features.ExtensionApiExpectTest)  Time elapsed: 0.115 sec  <<< FAILURE!

java.lang.RuntimeException: Uncompilable source code - Erroneous tree type: <any>

at org.jclouds.openstack.nova.v2_0.features.ExtensionApiExpectTest.testGetExtensionByAliasWhenResponseIs2xx(ExtensionApiExpectTest.java:95)

testGetExtensionByAliasWhenResponseIs404(org.jclouds.openstack.nova.v2_0.features.ExtensionApiExpectTest)  Time elapsed: 0.184 sec  <<< FAILURE!

java.lang.RuntimeException: Uncompilable source code - Erroneous tree type: <any>

at org.jclouds.openstack.nova.v2_0.features.ExtensionApiExpectTest.testGetExtensionByAliasWhenResponseIs404(ExtensionApiExpectTest.java:113)

testListExtensionsWhenResponseIs2xx(org.jclouds.openstack.nova.v2_0.features.ExtensionApiExpectTest)  Time elapsed: 0.184 sec  <<< FAILURE!

java.lang.RuntimeException: Uncompilable source code - Erroneous tree type: <any>

at org.jclouds.openstack.nova.v2_0.features.ExtensionApiExpectTest.testListExtensionsWhenResponseIs2xx(ExtensionApiExpectTest.java:59)

testListExtensionsWhenReponseIs404IsEmpty(org.jclouds.openstack.nova.v2_0.features.ExtensionApiExpectTest)  Time elapsed: 0.19 sec  <<< FAILURE!

java.lang.RuntimeException: Uncompilable source code - Erroneous tree type: <any>

at org.jclouds.openstack.nova.v2_0.features.ExtensionApiExpectTest.testListExtensionsWhenReponseIs404IsEmpty(ExtensionApiExpectTest.java:76)

Results :

Failed tests:   testGetExtensionByAliasWhenResponseIs2xx(org.jclouds.openstack.nova.v2_0.features.ExtensionApiExpectTest): Uncompilable source code - Erroneous tree type: <any>

  testGetExtensionByAliasWhenResponseIs404(org.jclouds.openstack.nova.v2_0.features.ExtensionApiExpectTest): Uncompilable source code - Erroneous tree type: <any>

  testListExtensionsWhenResponseIs2xx(org.jclouds.openstack.nova.v2_0.features.ExtensionApiExpectTest): Uncompilable source code - Erroneous tree type: <any>

  testListExtensionsWhenReponseIs404IsEmpty(org.jclouds.openstack.nova.v2_0.features.ExtensionApiExpectTest): Uncompilable source code - Erroneous tree type: <any>

[Adrian Cole]

P.s. I verified same failure as you had on my mac using jdk 7.  Skip tests for now, and will fix in the next day or two.

[Leander Bessa Beernaert]

Very well, anything I can do about the problem with the new method in the mean time?

[Adrian Cole]

If you run the live test for the method you just created in your ide (eclipse or whatever) then it will create the jclouds-wire.log which will show details that will help isolate the problem.

-A

[Leander Bessa Beernaert]

I tried to running the test on the compute-nova only, however I can't find the describe file in the target directory. Is that file generated only when global project is built?

I still haven't made any test methods, I wast just testing it out in some existing code. I'll see if I can prep a test method tomorrow.

[Adrian Cole]

ok. will look out for you then, dude.

On Mon, Nov 12, 2012 at 9:44 AM, Leander Bessa Beernaert

[Leander Bessa Beernaert]

Could you give me any pointers on how and where to implement this?  Let me elaborate. The method diagnostics is only supported by a limited set of hypervisors in OpenStack. The only ones I am aware of are XEN API and LibVirt (starting with the Folsom release). If this method is not present, the server will return a http 500 error code. Otherwise, a map of key and values will be returned containing certain hypervisor specific info. 

How should I tackle this in a test case? Do you have any dummy implementation of the OpenStack service?

[Adrian Cole]

Hi, Leander.

I was under the assumption you have access to a service that implements this.  If you do, then run the code you wrote in a test class (even a temporary one).  If the class that you wrote the test in is inside openstack-nova src tree, then logging will be setup via src/test/resources/logback.xml. in this case target/test-data/jclouds-wire.log should be around and you can send the trace to us.

If you only have access to a server that doesn't support the feature, then we should be able to test the parser that makes a 500 error coerce to null. 

-A
P.S. did you open a nova bug to make this return a sane error code in 400 range instead of 500?

[Leander Bessa Beernaert]

No no, I do have access to a Folsom installation of OpenStack. I only mentioned that since a reasonable test case should contemplate both options. 

I shall see if i can locate that file in my test app. Does it require a working log4j, I always get a NO-OP message?

Well, It only returns 500 if that operation is not supported, since it results in  crash on the server I think. Otherwise it returns sane codes. So I did not open any bug report.

[Adrian Cole]

Definitely open a bug as a user shouldnt be able to make a crash :)

Do you know anyone who has access to this feature that can send you the corresponding http request response, even if in curl? ( don't recall if I asked this before.. sorry if repeat ).

For project setup with logging, you can mine jclouds-examples repo for compute-basics

[Everett Toews]

You can also check out http://www.jclouds.org/documentation/reference/jclouds-logging/ for info on how to log.

Everett

[Leander Bessa Beernaert]

I'm afraid I do not. Although anyone with an installation of OpenStack Essex running libvirt should be able to reproduce the error. The api requests is endpoint/[server]/diagnostics.

Here's a verbose output of the diagonstics command in Folsom with libvirt:

REQ: curl -i http://10.0.107.2:8774/v2/4ebf76425efa4d01bc54a182185444a6/servers/0447a4df-36f3-42c8-8e56-a371867c8a87/diagnostics -X GET -H "X-Auth-Project-Id: project" -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: 05e818f7886a4a36a92f3aa24cc79543"

send: u'GET /v2/4ebf76425efa4d01bc54a182185444a6/servers/0447a4df-36f3-42c8-8e56-a371867c8a87/diagnostics HTTP/1.1\r\nHost: 10.0.107.2:8774\r\nx-auth-project-id: project\r\nx-auth-token: 05e818f7886a4a36a92f3aa24cc79543\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n'

reply: 'HTTP/1.1 200 OK\r\n'

header: X-Compute-Request-Id: req-5d87cb4d-7c4f-4580-85d8-dfca0a026cae

header: Content-Type: application/json

header: Content-Length: 331

header: Date: Tue, 13 Nov 2012 16:18:15 GMT

RESP:{'status': '200', 'content-length': '331', 'content-location': u'http://10.0.107.2:8774/v2/4ebf76425efa4d01bc54a182185444a6/servers/0447a4df-36f3-42c8-8e56-a371867c8a87/diagnostics', 'x-compute-request-id': 'req-5d87cb4d-7c4f-4580-85d8-dfca0a026cae', 'date': 'Tue, 13 Nov 2012 16:18:15 GMT', 'content-type': 'application/json'} {"vnet0_tx_errors": 0, "vda_read": 77364736, "vda_write": 237057024, "vnet0_tx_packets": 5390, "vda_write_req": 25911, "cpu0_time": 84320000000, "vnet0_tx": 937435, "vnet0_rx_drop": 0, "vda_errors": -1, "vnet0_rx_errors": 0, "memory": 524288, "vnet0_rx_packets": 6264, "vda_read_req": 9551, "vnet0_rx": 1001416, "vnet0_tx_drop": 0}

+------------------+-------------+

| Property         | Value       |

+------------------+-------------+

| cpu0_time        | 84320000000 |

| memory           | 524288      |

| vda_errors       | -1          |

| vda_read         | 77364736    |

| vda_read_req     | 9551        |

| vda_write        | 237057024   |

| vda_write_req    | 25911       |

| vnet0_rx         | 1001416     |

| vnet0_rx_drop    | 0           |

| vnet0_rx_errors  | 0           |

| vnet0_rx_packets | 6264        |

| vnet0_tx         | 937435      |

| vnet0_tx_drop    | 0           |

| vnet0_tx_errors  | 0           |

| vnet0_tx_packets | 5390        |

+------------------+-------------+

[Adrian Cole]

Hi, Leander.

Take a look at ServerApiExpectTest.  You should be able to make a test with this output similar to the others.

Once that works, you can add to ServerApiLive test.

Make sense?
-A

[Leander Bessa Beernaert]

I'm having trouble adding the log4j.xml to the classpath. Whatever i seem to do, it always displays the message:

log4j:WARN No appenders could be found for logger (org.jclouds.rest.internal.AsyncRestClientProxy).

log4j:WARN Please initialize the log4j system properly.

log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Is there any way i can point log4j to a specific file instead?

[Adrian Cole]

I'd recommend testing with slf4j/logback for now, as it is
preconfigured in both apis/openstack-nova and also compute-basics
example. we can help with log4j, but faster to just use what's
already there.

-a

On Tue, Nov 13, 2012 at 9:13 AM, Leander Bessa Beernaert

[Leander Bessa Beernaert]

I've tried again with slf4j and I still can't get any output to be generated :/.

I've added these two entries to my pom.xml and tried to add the example logback.xml to the project's classpath (although i'm not sure that worked). Am I missing something? 

         <dependency>

            <groupId>org.jclouds.driver</groupId>

            <artifactId>jclouds-slf4j</artifactId>

            <version>1.6.0-SNAPSHOT</version>

            <type>jar</type>

        </dependency>

        <dependency>

            <groupId>org.slf4j</groupId>

            <artifactId>slf4j-simple</artifactId>

            <version>1.7.2</version>

        </dependency>

[Adrian Cole]

Did you add new SLF4JLoggingModule() to contextBuilder.modules()?

[Leander Bessa Beernaert]

Yes i did:

Iterable<Module> modules = ImmutableSet.<Module>of(

                        new SLF4JLoggingModule());

context = ContextBuilder.newBuilder(PROVIDER).credentials(auth, password).

                        endpoint(endpoint).

                        modules(modules).

                        buildView(ComputeServiceContext.class);

[Adrian Cole]

Looks like you are missing logback as a dependency.  Can you look at compute-basics pom?  It should be in there.

[Leander Bessa Beernaert]

Okay, that solved it! I can now look at the logs :D. 

I'm still getting  null as a return value.

This is output of the wire.log:

2012-11-14 16:20:35,752 DEBUG [jclouds.wire] [i/o thread 0] << "{"access": {"token": {"expires": "2012-11-15T16:20:35Z", "id": "c889d20efe25424b9004441d9e6c8495", "tenant": {"enabled": true, "description": null, "name": "project", "id": "4ebf76425efa4d01bc54a182185444a6"}}, "serviceCatalog": [{"endpoints": [{"adminURL": "http://10.0.107.2:8776/v1/4ebf76425efa4d01bc54a182185444a6", "region": "nova", "internalURL": "http://10.0.107.2:8776/v1/4ebf76425efa4d01bc54a182185444a6", "id": "8d33cd77bef24625be0d88f77bf8a03a", "publicURL": "http://10.0.107.2:8776/v1/4ebf76425efa4d01bc54a182185444a6"}], "endpoints_links": [], "type": "volume", "name": "volume"}, {"endpoints": [{"adminURL": "http://10.0.107.2:9292/v1", "region": "nova", "internalURL": "http://10.0.107.2:9292/v1", "id": "c30a6d9580b6441ba9100ad04e2f2b28", "publicURL": "http://10.0.107.2:9292/v1"}], "endpoints_links": [], "type": "image", "name": "glance"}, {"endpoints": [{"adminURL": "http://10.0.107.2:8774/v2/4ebf76425efa4d01bc54a182185444a6", "region": "nova", "internalURL": "http://10.0.107.2:8774/v2/4ebf76425efa4d01bc54a182185444a6", "id": "fd84d42b17124909aeaa3227860f0cef", "publicURL": "http://10.0.107.2:8774/v2/4ebf76425efa4d01bc54a182185444a6"}], "endpoints_links": [], "type": "compute", "name": "nova"}, {"endpoints": [{"adminURL": "http://10.0.107.2:35357/v2.0", "region": "nova", "internalURL": "http://10.0.107.2:5000/v2.0", "id": "f02a45aa71214e36850540e9e513a944", "publicURL": "http://10.0.107.2:5000/v2.0"}], "endpoints_links": [], "type": "identity", "name": "keystone"}], "user": {"username": "admin", "roles_links": [], "id": "452ac7f1a2eb466d8e821e7baed455a5", "roles": [{"name": "admin"}], "name": "admin"}, "metadata": {"is_admin": 0, "roles": ["d54a6c0774c3432f96ad7686456f7301"]}}}"

2012-11-14 16:20:36,239 DEBUG [jclouds.wire] [i/o thread 0] >> "{}"

2012-11-14 16:20:36,239 DEBUG [jclouds.headers] [i/o thread 0] >> GET http://10.0.107.2:8774/v2/4ebf76425efa4d01bc54a182185444a6/servers/0447a4df-36f3-42c8-8e56-a371867c8a87/diagnostics HTTP/1.1

2012-11-14 16:20:36,239 DEBUG [jclouds.headers] [i/o thread 0] >> X-Auth-Token: c889d20efe25424b9004441d9e6c8495

2012-11-14 16:20:36,239 DEBUG [jclouds.headers] [i/o thread 0] >> Content-Type: application/json

2012-11-14 16:20:36,239 DEBUG [jclouds.headers] [i/o thread 0] >> Content-Length: 2

2012-11-14 16:20:36,281 DEBUG [jclouds.headers] [i/o thread 0] << HTTP/1.1 404 Not Found

2012-11-14 16:20:36,281 DEBUG [jclouds.headers] [i/o thread 0] << Date: Wed, 14 Nov 2012 16:20:36 GMT

2012-11-14 16:20:36,281 DEBUG [jclouds.headers] [i/o thread 0] << Connection: keep-alive

2012-11-14 16:20:36,281 DEBUG [jclouds.headers] [i/o thread 0] << X-Compute-Request-Id: req-c97a3c65-6dc8-4bbc-b16d-24184765fc9c

2012-11-14 16:20:36,281 DEBUG [jclouds.headers] [i/o thread 0] << Content-Type: application/json; charset=UTF-8

2012-11-14 16:20:36,282 DEBUG [jclouds.headers] [i/o thread 0] << Content-Length: 78

2012-11-14 16:20:36,282 DEBUG [jclouds.wire] [i/o thread 0] << "{"itemNotFound": {"message": "The resource could not be found.", "code": 404}}"

And this is the verbose output of the python command line tool:

RESP:{'status': '200', 'content-length': '1448', 'content-location': u'http://10.0.107.2:8774/v2/4ebf76425efa4d01bc54a182185444a6/servers/0447a4df-36f3-42c8-8e56-a371867c8a87', 'x-compute-request-id': 'req-4bc5bb37-7818-4535-af4a-3da7048ce839', 'date': 'Wed, 14 Nov 2012 16:32:28 GMT', 'content-type': 'application/json'} {"server": {"status": "ACTIVE", "updated": "2012-11-12T10:19:49Z", "hostId": "459f0ecc2f3b53cdc1b90d0e7084f09a7439e7ecb9b9bf91c14a1867", "OS-EXT-SRV-ATTR:host": "cloud31", "addresses": {"project_network": [{"version": 4, "addr": "10.0.108.3"}]}, "links": [{"href": "http://10.0.107.2:8774/v2/4ebf76425efa4d01bc54a182185444a6/servers/0447a4df-36f3-42c8-8e56-a371867c8a87", "rel": "self"}, {"href": "http://10.0.107.2:8774/4ebf76425efa4d01bc54a182185444a6/servers/0447a4df-36f3-42c8-8e56-a371867c8a87", "rel": "bookmark"}], "key_name": null, "image": {"id": "7957422a-5e6d-4a2a-bb01-1cfec2d2ec27", "links": [{"href": "http://10.0.107.2:8774/4ebf76425efa4d01bc54a182185444a6/images/7957422a-5e6d-4a2a-bb01-1cfec2d2ec27", "rel": "bookmark"}]}, "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "active", "OS-EXT-SRV-ATTR:instance_name": "instance-00000229", "OS-EXT-SRV-ATTR:hypervisor_hostname": "cloud31.cluster.lsd.di.uminho.pt", "flavor": {"id": "1", "links": [{"href": "http://10.0.107.2:8774/4ebf76425efa4d01bc54a182185444a6/flavors/1", "rel": "bookmark"}]}, "id": "0447a4df-36f3-42c8-8e56-a371867c8a87", "security_groups": [{"name": "default"}], "user_id": "452ac7f1a2eb466d8e821e7baed455a5", "name": "Test1", "created": "2012-11-12T10:19:21Z", "tenant_id": "4ebf76425efa4d01bc54a182185444a6", "OS-DCF:diskConfig": "MANUAL", "accessIPv4": "", "accessIPv6": "", "progress": 0, "OS-EXT-STS:power_state": 1, "config_drive": "", "metadata": {}}}

REQ: curl -i http://10.0.107.2:8774/v2/4ebf76425efa4d01bc54a182185444a6/servers/0447a4df-36f3-42c8-8e56-a371867c8a87/diagnostics -X GET -H "X-Auth-Project-Id: project" -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: 5a4cae2c68f547cdb16d6cdc944ab14d"

send: u'GET /v2/4ebf76425efa4d01bc54a182185444a6/servers/0447a4df-36f3-42c8-8e56-a371867c8a87/diagnostics HTTP/1.1\r\nHost: 10.0.107.2:8774\r\nx-auth-project-id: project\r\nx-auth-token: 5a4cae2c68f547cdb16d6cdc944ab14d\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n'

reply: 'HTTP/1.1 200 OK\r\n'

header: X-Compute-Request-Id: req-a0ee739a-883d-4633-8405-6a9e98e38591

header: Content-Type: application/json

header: Content-Length: 334

header: Date: Wed, 14 Nov 2012 16:32:29 GMT

RESP:{'status': '200', 'content-length': '334', 'content-location': u'http://10.0.107.2:8774/v2/4ebf76425efa4d01bc54a182185444a6/servers/0447a4df-36f3-42c8-8e56-a371867c8a87/diagnostics', 'x-compute-request-id': 'req-a0ee739a-883d-4633-8405-6a9e98e38591', 'date': 'Wed, 14 Nov 2012 16:32:29 GMT', 'content-type': 'application/json'} {"vnet0_tx_errors": 0, "vda_read": 77364736, "vda_write": 415446016, "vnet0_tx_packets": 9701, "vda_write_req": 47278, "cpu0_time": 143150000000, "vnet0_tx": 1691221, "vnet0_rx_drop": 0, "vda_errors": -1, "vnet0_rx_errors": 0, "memory": 524288, "vnet0_rx_packets": 11271, "vda_read_req": 9551, "vnet0_rx": 1805288, "vnet0_tx_drop": 0}

I've check the request URIs and they are both the same, I have no idea wh jclouds gets a 404 response :s

[Adrian Cole]

Awesome.

Can you look at ServerApiExpectTest, add a test for this with the data below and push your work in progress to a branch on your fork?

We can help easier this way.

-A

...

[Leander Bessa Beernaert]

I'll do it first thing tomorrow :)

...

[Message clipped]  

[Leander Bessa Beernaert]

I've added my modification to JClouds here: g...@github.com:LeanderBB/jclouds.git.

[Adrian Cole]

For those of us walking around with android and no git client... Can you send the github commit http url for what you did?

-A

P.S.

When you are unsure about a particular solution and want review, the easiest way (for me) is to use pull requests. Pull your change onto a branch, push it, then go to github and click pull request. In there, you can describe the situation and your proposed solution safely. You can rework the commit based on the feedback.

example commands..

git checkout -b fix-logging
git add (files you changed)
git commit -m "log missing in nova create server"
git push origin fix-logging
git checkout master (or whatever you were on)

// then go to github and click pull request, add description and cc people with @userid

if you have to change something or update it, you can rework the pull request like so

git checkout fix-logging
git rebase master (or whatever it was based on)
git reset HEAD~1 // unstages the files so you can redo the commit
git commit -a -m "logging missing on create server and delete server"
git push -f origin fix-logging
git checkout master (or whatever you were on)

Using a pattern like this is pretty normal, and keeps the history clean. Also, it allows you to get feedback and review on the change with very little overhead.

...

[Leander Bessa Beernaert]

Sorry about that. You can see the pull request at https://github.com/jclouds/jclouds/pull/995. For some unknown reason my description did not go through. I'm adding it now, should be up in just a bit.

...

[Message clipped]  

[Adrian Cole]

Thanks!

>>>>>>>> > jclouds-dev+unsubscribe@g

...