405 error when attempting to delete a snapshot

[syhR8]

Hello,

I've been using the Reaper http rest api to automate the cleanup of some snapshots and I'm hitting a problem. I receive a HTTP/1.1 405 Method Not Allowed error when attempting to delete a snapshot. I'm attempting to use the following method..

DELETE /snapshot/{clusterName}/{snapshotName}

But after a little investigation I can see it's attempting to use the following method...

DELETE /snapshot/{clusterName}/{host}/{snapshotName}

The reaper.log file tells me this...

GET /snapshot/pcs_cluster/bb311fb0-183a-11ea-8848-4717022ccff1] i.c.r.SnapshotResource - Failure when establishing JMX connection to bb311fb0-183a-11ea-8848-4717022ccff1:7199

io.cassandrareaper.ReaperException: Failure when establishing JMX connection to bb311fb0-183a-11ea-8848-4717022ccff1:7199

So it seems it's taking my snapshot as a hostname and trying to connect to it. The following curl can reproduce the error...

curl -v -X DELETE --cookie "JSESSIONID=XXXXXXXXXXXXXXX.node0;Path=/" http://xxxxxxxxxxxxxx:8080/snapshot/xxx_cluster/bb311fb0-183a-11ea-8848-4717022ccff1

* About to connect() to xxxxxxxx port 8080 (#0)
*   Trying 10.223.0.90...
* Connected to xxxxxxxx (X.X.X.X) port 8080 (#0)
> DELETE /snapshot/xxx_cluster/bb311fb0-183a-11ea-8848-4717022ccff1 HTTP/1.1
> User-Agent: curl/7.29.0
> Host: xxxxxxxx:8080
> Accept: */*
> Cookie: JSESSIONID=XXXXXXXXXXXXXXXXXXXX.node0;Path=/
>
< HTTP/1.1 405 Method Not Allowed
< Date: Mon, 09 Dec 2019 17:14:43 GMT
< Allow: POST,GET,OPTIONS
< Content-Type: application/json
< Content-Length: 52
<
* Connection #0 to host xxxxxxxx left intact

Response 

 

{"code":405,"message":"HTTP 405 Method Not Allowed"}

I am doing something wrong here or have I hit a bug?

I do actually have the ip addresses of all hosts with the snapshot so I could issue a call for each host if needed. Here is the Java stack trace from the reaper.log file...

ERROR  [2019-12-09 18:02:05,375] [dw-62158 - GET /snapshot/pcs_cluster/bb311fb0-183a-11ea-8848-4717022ccff1] i.c.r.SnapshotResource - Failure when establishing JMX connection to bb311fb0-183a-11ea-8848-4717022ccff1:7199
io.cassandrareaper.ReaperException: Failure when establishing JMX connection to bb311fb0-183a-11ea-8848-4717022ccff1:7199
            at io.cassandrareaper.jmx.JmxProxyImpl.connect(JmxProxyImpl.java:255)
            at io.cassandrareaper.jmx.JmxProxyImpl.connect(JmxProxyImpl.java:161)
            at io.cassandrareaper.jmx.JmxConnectionFactory$JmxConnectionProvider.apply(JmxConnectionFactory.java:227)
            at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1688)
            at io.cassandrareaper.jmx.JmxConnectionFactory.connectImpl(JmxConnectionFactory.java:104)
            at io.cassandrareaper.jmx.JmxConnectionFactory.connect(JmxConnectionFactory.java:126)
            at io.cassandrareaper.service.SnapshotService.listSnapshots(SnapshotService.java:153)
            at io.cassandrareaper.service.SnapshotService.listSnapshotsGroupedByName(SnapshotService.java:139)
            at io.cassandrareaper.resources.SnapshotResource.listSnapshots(SnapshotResource.java:158)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:498)
            at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
            at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
            at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
            at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160)
            at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
            at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
            at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
            at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
            at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
            at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
            at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
            at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
            at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
            at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
            at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
            at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
            at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
            at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
            at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
            at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
            at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
            at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
            at io.dropwizard.jetty.NonblockingServletHolder.handle(NonblockingServletHolder.java:49)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)
            at io.dropwizard.servlets.ThreadNameFilter.doFilter(ThreadNameFilter.java:35)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
            at io.dropwizard.jersey.filter.AllowedMethodsFilter.handle(AllowedMethodsFilter.java:45)
            at io.dropwizard.jersey.filter.AllowedMethodsFilter.doFilter(AllowedMethodsFilter.java:39)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
            at org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:311)
            at org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:265)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
            at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
            at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
            at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
            at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
            at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
            at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
            at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
            at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
            at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
            at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
            at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
            at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
            at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
            at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
            at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
            at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
            at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
            at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
            at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
            at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
            at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
            at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
            at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
            at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
            at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
            at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
            at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:239)
            at io.dropwizard.jetty.RoutingHandler.handle(RoutingHandler.java:52)
            at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:724)
            at io.dropwizard.jetty.BiDiGzipHandler.handle(BiDiGzipHandler.java:67)
            at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:169)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
            at org.eclipse.jetty.server.Server.handle(Server.java:531)
            at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
            at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
            at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
            at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
            at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
            at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
            at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
            at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
            at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ConfigurationException [Root exception is java.rmi.UnknownHostException: Unknown host: bb311fb0-183a-11ea-8848-4717022ccff1; nested exception is:
            java.net.UnknownHostException: bb311fb0-183a-11ea-8848-4717022ccff1]
            at java.util.concurrent.FutureTask.report(FutureTask.java:122)
            at java.util.concurrent.FutureTask.get(FutureTask.java:206)
            at io.cassandrareaper.jmx.JmxProxyImpl.connectWithTimeout(JmxProxyImpl.java:273)
            at io.cassandrareaper.jmx.JmxProxyImpl.connect(JmxProxyImpl.java:214)
            ... 97 common frames omitted

Cheers,

Rhys