PAXWEB 8.0.9 Jetty - Correct way to override default context

[Dave Smith]

What is the correct way to override the default servlet context? I am doing this...

@Component(service = ServletContextHelper.class,property = { Constants.SERVICE_RANKING+":Integer="+Integer.MAX_VALUE})
@HttpWhiteboardContext(name = "default",path = "/")
public class NoSecurityContextImpl extends ServletContextHelper
{

What I am seeing is if a servlet is registered BEFORE the new default context it just seems to disappear, anything after gets put in the new default. When I call HttpServiceRuntime.servletDTOs I see the original default context with no servlets attached to it and my overridden one with the servlets that were registered after. failedServletDTOs,failedServletContextDTOs are empty. If I remove my override the servlets appear.

What is weird is it looks like the servlet is getting put into the right context if I read the log correctly ...

Registering ServletModel{id=ServletModel-4,name='com.candatag.k8s.zoo.servlet.ReadyProbe',urlPatterns=[/osgi/started],contexts=[{WB,OCM-2,default,/}]}

INFO org.ops4j.pax.web.service.spi.servlet.OsgiServletContext - Unegistering OsgiServletContext{model=OsgiContextModel{WB,id=OCM-1,name='default',path='/',bundle=org.ops4j.pax.web.pax-web-extender-whiteboard,context=(supplier)}} as OSGi service for "/" context path

Registering OsgiServletContext{model=OsgiContextModel{WB,id=OCM-2,name='default',path='/',bundle=com.candatag.web.util,ref={org.osgi.service.http.context.ServletContextHelper}={service.id=57, osgi.http.whiteboard.context.name=default, service.bundleid=19, service.scope=bundle, service.ranking=2147483647, component.name=com.candatag.web.util.zoo.impl.NoSecurityContextImpl, osgi.http.whiteboard.context.path=/, component.id=22}}} as OSGi service for "/" context path

Dave

[Grzegorz Grzybek]

Hello

Actually, the initial reason of entire Pax Web 8 refactoring was the context handling - ability to register one servlet into multiple contexts and multiple servlets into one context.

There are really lot of integration tests that show exactly this.

I'd have to see your example and check what's the problem there - you seem to correctly override "default" whiteboard context with "/" path and higher ranking, so it should work.

https://github.com/ops4j/org.ops4j.pax.web/tree/web-8.0.9/samples/samples-whiteboard/whiteboard-ds is and example of SCR registration of multiple web elements and contexts.

This integration test (https://github.com/ops4j/org.ops4j.pax.web/blob/web-8.0.9/pax-web-itest/pax-web-itest-server/src/test/java/org/ops4j/pax/web/itest/server/whiteboard/WhiteboardAndHttpServiceTest.java#L198) - overridenDefaultContextsWithWhiteboardServlet() shows how "default" + "/" context is overriden.

Could you please share your project? It can be attached to an issue in https://github.com/ops4j/org.ops4j.pax.web/issues

kind regards

Grzegorz Grzybek

--
--
------------------
OPS4J - http://www.ops4j.org - op...@googlegroups.com

---
You received this message because you are subscribed to the Google Groups "OPS4J" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ops4j+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ops4j/CA%2BFCLu2SbiSKmN7W3-ZQqHLYCA67vF104UCVWtU77-63tcarUA%40mail.gmail.com.

[Dave Smith]

It will be a little tricky to send my whole project in , let me see what I can do , however I have reviewed the test cases in the link provided and it looks like you are not covering my case ...

I do not see a test where  ...

Register the default handler ->   HttpContext defaultContext = wc.createDefaultHttpContext(); , not sure if this also creates a ServletContextHelper wrapper as well

Then create a servlet that attaches to it , but do not call the servlet , just check it is added

Register the default override with a high service ranking .

See if this happens

org.ops4j.pax.web.service.spi.servlet.OsgiServletContext - Unegistering OsgiServletContext{model=OsgiContextModel{WB,id=OCM-1,name='default',path='/'

org.ops4j.pax.web.service.spi.servlet.OsgiServletContext - Registering OsgiServletContext{model=OsgiContextModel{WB,id=OCM-2,name='default',path='/',

And then see if the servlet is in the new context ...

The override testcases seem to all create the override first before the first servlet is registered OR you are changing the path in the default context ...

Dave

To view this discussion on the web visit https://groups.google.com/d/msgid/ops4j/CAAdXmhrMU6_5ABFc0oxevK%2BjuZVhOzxdK0cqvyTz6pCWZAM7Yg%40mail.gmail.com.

[dave....@candata.com]

My other question would be if you are "Unregistering " the default context should it still show up in the HttpServiceRuntime.getRuntimeDTO().servletContextDTOs ? It does now

Dave

[dave....@candata.com]

Still trying to get a test case but I do believe it is a threading issue ... What I find strange is the servlet that disappears gets registered like this

[paxweb-config-1-thread-1] INFO org.ops4j.pax.web.service.internal.HttpServiceEnabled - Registering ServletModel{id=ServletModel-4,name='com.candatag.k8s.zoo.servlet.ReadyProbe',urlPatterns=[/osgi/started],contexts=[{WB,OCM-2,default,/}]}

Receiving Batch{"Registration of ServletModel{id=ServletModel-4,name='com.candatag.k8s.zoo.servlet.ReadyProbe',urlPatterns=[/osgi/started],contexts=[{WB,OCM-2,default,/}]}", size=1}

Adding servlet ServletModel{id=ServletModel-4,name='com.candatag.k8s.zoo.servlet.ReadyProbe',urlPatterns=[/osgi/started],contexts=[{WB,OCM-2,default,/}]}

Now what is ODD is that OCM-2 is not registered yet!  It follows in the logs

 Receiving Batch{"Registration of OsgiContextModel{WB,id=OCM-2,name='default',path='/',bundle=com.candatag.web.util,ref={org.osgi.service.http.context.ServletContextHelper}={service.id=39, osgi.http.whiteboard.context.name=default, service.bundleid=17, service.scope=bundle, service.ranking=2147483647, component.name=com.candatag.web.util.zoo.impl.NoSecurityContextImpl, osgi.http.whiteboard.context.path=/, component.id=1}}", size=1}

So how does the servlet know that this new default context is coming?  In my test harness I see the servlet unregistering and then registering again when the context changes...

It looks like we are half pregnant, somebody knows the context is  there but it is in an unregistered state so the servlet modal does not get attached.

Dave

[Grzegorz Grzybek]

Hello

It's all loosely coupled - a "servlet registration" knows the LDAP filter (by default "osgi.http.whiteboard.context.select=(osgi.http.whiteboard.context.name=default)") for it's contexts. And each time the new context is registered, the web elements with matching filter are being re-registered.

The fact that you're overriding an "OSGi Context" (ServletContextHelper) within the same context path "/" means that the servlet context is restarted (because there's higher-ranked OsgiServletContext), but the servlet should be simply kept in existing "/" ServletContext. I know - lots of contexts.

Today I was checking few other https://github.com/ops4j/org.ops4j.pax.web/issues, but I remember about your scenario - I should have an explanation tomorrow.

regards

Grzegorz Grzybek

To view this discussion on the web visit https://groups.google.com/d/msgid/ops4j/e972d47a-f8b3-4dec-9ba1-94549e29a250n%40googlegroups.com.

[Dave Smith]

I have been doing a little more testing. I have an activator method like this ...

Thread context = new Thread(()->{
Hashtable<String, Object> aDic = new Hashtable<>();
aDic.put(HttpWhiteboardConstants.HTTP_WHITEBOARD_CONTEXT_NAME, "default");
aDic.put(HttpWhiteboardConstants.HTTP_WHITEBOARD_CONTEXT_PATH, "/");
aDic.put(Constants.SERVICE_RANKING,Integer.MAX_VALUE);
ctx.registerService(ServletContextHelper.class, new DefaultContxt(), aDic);

});

CountDownLatch latch = new CountDownLatch(1);
Thread s2 = new Thread(()->{

Hashtable<String, Object> aDic = new Hashtable<>();
aDic.put(HttpWhiteboardConstants.HTTP_WHITEBOARD_SERVLET_NAME, "test2");
aDic.put(HttpWhiteboardConstants.HTTP_WHITEBOARD_SERVLET_PATTERN, "/test2");
ctx.registerService(Servlet.class, new TestServlet("Test 2"), aDic);
latch.countDown();
});

s2.start();
latch.await();
Thread.sleep(400);
context.start();

On my machine any value where sleep is <400 the context is always started first otherwise the servlet gets started first and gets unresgtered and re-registered into the new context. I am guessing it takes longer to init the servlet and I wonder if the Context is getting updated during the servlet init process. (That is what it would seem like from the logs)...

Let me know if you need any more info ...

Dave

To view this discussion on the web visit https://groups.google.com/d/msgid/ops4j/CAAdXmhqzknu%3DwfrL5d-hWbHe%2BGa%2Bazbx6Yg__OTuC683B8VbMA%40mail.gmail.com.

[Grzegorz Grzybek]

Hi. See inline

pon., 19 wrz 2022 o 20:21 Dave Smith <dave....@candata.com> napisał(a):

I have been doing a little more testing. I have an activator method like this ...

Thread context = new Thread(()->{
Hashtable<String, Object> aDic = new Hashtable<>();
aDic.put(HttpWhiteboardConstants.HTTP_WHITEBOARD_CONTEXT_NAME, "default");
aDic.put(HttpWhiteboardConstants.HTTP_WHITEBOARD_CONTEXT_PATH, "/");
aDic.put(Constants.SERVICE_RANKING,Integer.MAX_VALUE);
ctx.registerService(ServletContextHelper.class, new DefaultContxt(), aDic);

});

CountDownLatch latch = new CountDownLatch(1);
Thread s2 = new Thread(()->{

Hashtable<String, Object> aDic = new Hashtable<>();
aDic.put(HttpWhiteboardConstants.HTTP_WHITEBOARD_SERVLET_NAME, "test2");
aDic.put(HttpWhiteboardConstants.HTTP_WHITEBOARD_SERVLET_PATTERN, "/test2");
ctx.registerService(Servlet.class, new TestServlet("Test 2"), aDic);
latch.countDown();
});

s2.start();
latch.await();
Thread.sleep(400);
context.start();

On my machine any value where sleep is <400 the context is always started first otherwise the servlet gets started first and gets unresgtered and re-registered into the new context. I am guessing it takes longer to init the servlet and I wonder if the Context is getting updated during the servlet init process. (That is what it would seem like from the logs)...

Let me know if you need any more info ...

I'll check the order of operations tomorrow. The point is that in Pax Web 8 I've created several such tests, including SCR one (https://github.com/ops4j/org.ops4j.pax.web/tree/web-8.0.9/samples/samples-whiteboard/whiteboard-ds - see the readme describing fundamental difficulty with SCR) where contexts and servlets are registered, but eventually everything is fine.

So a question - is your servlet eventually responding to a GET request? Don't just look at the logs for registration/unregistration of the context. The lines you've sent:

Unegistering OsgiServletContext{model=OsgiContextModel{WB,id=OCM-1,name='default',path='/',bundle=org.ops4j.pax.web.pax-web-extender-whiteboard,context=(supplier)}} as OSGi service for "/" context path

Registering OsgiServletContext{model=OsgiContextModel{WB,id=OCM-2,name='default',path='/',bundle=com.candatag.web.util,ref={org.osgi.service.http.context.ServletContextHelper}={service.id=57, osgi.http.whiteboard.context.name=default, service.bundleid=19, service.scope=bundle, service.ranking=2147483647, component.name=com.candatag.web.util.zoo.impl.NoSecurityContextImpl, osgi.http.whiteboard.context.path=/, component.id=22}}} as OSGi service for "/" context path

are information that different object of OsgiServletContext class is registered as the OSGi service of javax.servlet.ServletContext interface. This is according to chapter 128.3.4 (https://docs.osgi.org/specification/osgi.cmpn/7.0.0/service.war.html#i3078599) of Web Applications Specification, but has nothing to do with Whiteboard - the underlying Jetty/Tomcat/Undertow context stays started.

regards

Grzegorz Grzybek

 

To view this discussion on the web visit https://groups.google.com/d/msgid/ops4j/CA%2BFCLu1OssPACzuqNDCpxT3AGz%3DRjarNubJUi%2BPXuog2D0aWMA%40mail.gmail.com.

[Dave Smith]

Never responds to the GET request.  I have a helper method that displays the ServletDTO. It does not show up there either. It does seem to be a timing issue.

I have attached a couple of log dumps. The checks of the DTO (They are the last lines of the log entries) happen about 10 seconds after the container starts.

In server.txt there is one servlet that gets dropped from the new default context

In some.txt there are 2 servlets that get dropped and then there are 2 that are registered successfully .

 Dave

To view this discussion on the web visit https://groups.google.com/d/msgid/ops4j/CAAdXmhrP%3DUawM0_EYyg%2B3NuyMDjz%3DRzqvAP%3D%3DfTkcXUXgJH6jw%40mail.gmail.com.

[Grzegorz Grzybek]

Hello Dave

Your logs are really helpful, I've found (in some.txt) that a servlet named "com.candatag.repository.servlet.RepoMenu" is registered into OCM-3 (/admin):

[paxweb-config-1-thread-1] INFO org.ops4j.pax.web.service.internal.HttpServiceEnabled - Registering ServletModel{id=ServletModel-9,name='com.candatag.repository.servlet.RepoMenu',urlPatterns=[/repo],contexts=[{WB,OCM-3,com.candatag.web.security.admin.services.AdminWebSecurity,/admin}]}

but only later this OCM-3 is passed to Jetty:

[paxweb-config-1-thread-1] INFO org.ops4j.pax.web.service.jetty.internal.JettyServerController - Receiving Batch{"Registration of OsgiContextModel{WB,id=OCM-3,name='com.candatag.web.security.admin.services.AdminWebSecurity',path='/admin',bundle=com.candatag.web.security.admin,ref={org.osgi.service.http.context.ServletContextHelper, com.candatag.web.security.admin.services.AdminWebSecurity}={service.id=55, osgi.http.whiteboard.context.name=com.candatag.web.security.admin.services.AdminWebSecurity, service.bundleid=18, service.scope=bundle, component.name=com.candatag.web.security.admin.zoo.impl.AdminContext, osgi.http.whiteboard.context.path=/admin, component.id=21}}", size=2}
[paxweb-config-1-thread-1] INFO org.ops4j.pax.web.service.jetty.internal.JettyServerWrapper - Creating new Jetty context for ServletContextModel{id=ServletContextModel-12,contextPath='/admin'}
[paxweb-config-1-thread-1] INFO org.ops4j.pax.web.service.jetty.internal.JettyServerWrapper - Adding OsgiContextModel{WB,id=OCM-3,name='com.candatag.web.security.admin.services.AdminWebSecurity',path='/admin',bundle=com.candatag.web.security.admin,ref={org.osgi.service.http.context.ServletContextHelper, com.candatag.web.security.admin.services.AdminWebSecurity}={service.id=55, osgi.http.whiteboard.context.name=com.candatag.web.security.admin.services.AdminWebSecurity, service.bundleid=18, service.scope=bundle, component.name=com.candatag.web.security.admin.zoo.impl.AdminContext, osgi.http.whiteboard.context.path=/admin, component.id=21}} to o.o.p.w.s.j.i.PaxWebServletContextHandler@48beedf5{/admin,null,STOPPED}

I'll ensure that this doesn't lead to problems.

I'll check further, bu if you could please set DEBUG log level for entire "org.ops4j.pax.web" logger and add timestamps, we'd get more information.

In server.txt I think everything looks fine though (if I read the DTOs correctly):

Servlet Context {"name":"default", "contextPath":"/", ..., "serviceId":39, "servletDTOs":[{"patterns":["/osgi/started"], ..., "name":"com.candatag.k8s.zoo.servlet.ReadyProbe", ...], ...}
 - {"patterns":["/osgi/started"], ..., "name":"com.candatag.k8s.zoo.servlet.ReadyProbe", ...}

Servlet Context {"name":"default", "contextPath":"/", ..., "serviceId":0, "servletDTOs":[], ...}
 - <no servlets>

Servlet Context {"name":"com.candatag.web.security.admin.services.AdminWebSecurity", "contextPath":"/admin", ..., "serviceId":40, "servletDTOs":[
 - {"patterns":["/repo/localservice/console/services/view"], ..., "name":"com.candatag.osgi.console.servlet.ViewService", ...},
 - {"patterns":["/repo/localservice/console/services"], ..., "name":"com.candatag.osgi.console.servlet.Services", ...},
 - {"patterns":["/repo/localservice/console/services/check"], ..., "name":"com.candatag.osgi.console.servlet.SystemCheck", ...},
 - {"patterns":["/repo/localservice/console/run"], ..., "name":"com.candatag.osgi.console.servlet.RunService", }
], ...}

so "/osgi/started" servlet is registered to "/" context, which is (from Whiteboard point of view) "backed" by OCM-3 (serviceId=39) - the one registered by you, not the default one from pax-web-extender-whiteboard.

However, in "some.txt", I see what I mentioned - that a servlet registration is being handled before OCM-3 is processed. Indeed, in code, the OsgiContextModel is added to Whiteboard internal structures, before it's passed further. And in between, a servlet may refer to it. Checking.

regards

Grzegorz Grzybek

To view this discussion on the web visit https://groups.google.com/d/msgid/ops4j/CA%2BFCLu3Wm1mXtGYBGHr7MoLpo83%2BQbd0TMHNheKiWii_at-CsQ%40mail.gmail.com.

[Grzegorz Grzybek]

Anyway, I've created a Pax Web issue at https://github.com/ops4j/org.ops4j.pax.web/issues/1769, so we can continue the discussion there.

regards

Grzegorz Grzybek

[Grzegorz Grzybek]

Please check the activator I used: https://github.com/ops4j/org.ops4j.pax.web/commit/c4efc95ebe1e5ecd4d7763f3f04267f9213a845d#diff-c7a6d3e304fd5dd93a853e2d86a1f1ea390f1f1d313691af2854a63b527ccaa4

regards

Grzegorz Grzybek

[Grzegorz Grzybek]

I've changed the activator, so the servlet is registered 400ms before the context - consistently the test passes when invoking the servlet - the new context is ALWAYS used and I see this in logs:

11:57:22.032 [paxweb-config-1-thread-1] INFO  (HttpServiceEnabled.java:592) org.ops4j.pax.web.service.internal.HttpServiceEnabled - Unregistering servlet model "ServletModel{id=ServletModel-2,name='whiteboard-servlet',urlPatterns=[/wb/*],contexts=[{WB,OCM-1,default,/}]}"
...

11:57:22.039 [main] DEBUG (WhiteboardExtenderContext.java:494) org.ops4j.pax.web.extender.whiteboard.internal.WhiteboardExtenderContext - Registering ServletModel{id=ServletModel-2,name='whiteboard-servlet',urlPatterns=[/wb/*],contexts=[{WB,OCM-5,default,/}]} again after its context selection filter matched new set of contexts

kind regards

Grzegorz Grzybek

[Dave Smith]

As I suspected tuning on the  debugging added extra delays and thus I am now able to get logs where it works and does not. I will move the conversation to the ticket. I have updated it with logs from 2 runs. One where it starts successfully and one where it drops the servlet. Looks like we have a nasty threading bug ...

Dave

To view this discussion on the web visit https://groups.google.com/d/msgid/ops4j/CAAdXmhrPvxRF829dtYoEZOLHhj%3DtwJ15VV0KA0SK0jxjZn9ZMw%40mail.gmail.com.

[dave....@candata.com]

I see you have committed a fix. Are you planning a release? Or should I build from source to test?

Dave

[Grzegorz Grzybek]

Hello

I plan a release very soon. But in the meantime you can test if it works for you ;)

I just saw new Tomcat release, which I want to include, but the release should be ready this week.

There were some TCCL related issues (one fixed, caused two other issues, then reverted), but it's fine now ;)

regards

Grzegorz Grzybek

To view this discussion on the web visit https://groups.google.com/d/msgid/ops4j/a71ebdd2-22c1-4e0d-9e01-512f16d24dden%40googlegroups.com.

[dave....@candata.com]

Just downloaded the latest. So far so good. Will report back if I find any issues...

[Grzegorz Grzybek]

Hello

śr., 28 wrz 2022 o 14:36 dave....@candata.com <dave....@candata.com> napisał(a):

Just downloaded the latest. So far so good. Will report back if I find any issues...

Whew! Thanks for checking - I was waiting for your feedback ;)

regards

Grzegorz Grzybek

 

To view this discussion on the web visit https://groups.google.com/d/msgid/ops4j/a7ef6266-fe73-40a4-a0b7-b9bad25dc8b1n%40googlegroups.com.