[PAX-LOGGING] Log4J2 problem

46 views
Skip to first unread message

Grzegorz Grzybek

unread,
May 22, 2019, 11:21:27 AM5/22/19
to op...@googlegroups.com, d...@karaf.apache.org
Hello

(sorry for writing to two mailing lists, but I think it's important).

I've just found nasty problem.

After having lots of fun with pax-logging-service and pax-logging-logback I wanted to clean up pax-logging-log4j2. But I found that original, already available pax-logging-log4j2 bundle actually has Export-Package header (pax-logging-service and pax-logging-logback don't export anything).

The strange thing was that all bundles have:

Export-Package: \
 !*

The difference is that pax-logging-log4j2 additionally has:

Private-Package: \
...
META-INF; -split-package:=merge-first, \
...

So it ... took the META-INF/MANIFEST.MF from org.apache.logging.log4j:log4j-core. It wouldn't be a problem if pax-logging-log4j2 exported something - even single package.

Here's why (aQute.lib.osgi.Analyzer#calcManifest()):

// Copy old values into new manifest, when they
// exist in the old one, but not in the new one
merge(manifest, dot.getManifest());

pax-logging-log4j2 bundle didn't have any Export-Package (as intended), so it just inherited it from org.apache.logging.log4j:log4j-core (definitely *not* as intended...).

This is related to https://ops4j1.jira.com/browse/PAXLOGGING-240 and the discussion here[1].

To be honest, the only thing I think is sensible here is to stop exporting anything from pax-logging-log4j2... Extensions should be done via fragments or pax-logging-api's org.ops4j.pax.logging.spi interfaces (like PaxAppender).

WDYT?

regards
Grzegorz Grzybek
===

Christoph Läubrich

unread,
May 23, 2019, 2:50:11 AM5/23/19
to op...@googlegroups.com
WHat is teh reason for merging the META-INF at all?
Beside that why embeeed log4j at all if it is already an OSGi Bundle?

I think that people do embeed a way to much stuff in the OSGi world that
does not make sense. Embedding should only be used if the lib is not an
OSGi-Jar (where I prefere to open a PR on the project ot add the
required headers).
In all other cases a proper version import does the job much better
beside the fact that embedding might cuase problems with licensing and
maintaining stuff (e.g. security updates).

Am 22.05.19 um 17:21 schrieb Grzegorz Grzybek:
> Hello
>
> (sorry for writing to two mailing lists, but I think it's important).
>
> I've just found nasty problem.
>
> After having lots of fun with pax-logging-service and
> pax-logging-logback I wanted to clean up pax-logging-log4j2. But I found
> that original, already available pax-logging-log4j2 bundle actually has
> Export-Package header (pax-logging-service and pax-logging-logback don't
> export anything).
>
> The strange thing was that all bundles have:
>
> Export-Package: \
>  !*
>
> The difference is that pax-logging-log4j2 additionally has
> <https://github.com/ops4j/org.ops4j.pax.logging/blob/b8c9137/pax-logging-log4j2/osgi.bnd#L5>:
>
> Private-Package: \
> ...
> META-INF; -split-package:=merge-first, \
> ...
>
> So it ... took the META-INF/MANIFEST.MF from
> org.apache.logging.log4j:log4j-core. It wouldn't be a problem if
> pax-logging-log4j2 exported something - even single package.
>
> Here's why
> <https://github.com/bndtools/bnd/blob/4.2.0.REL/biz.aQute.bndlib/src/aQute/bnd/osgi/Analyzer.java#L1063-L1065>
> (aQute.lib.osgi.Analyzer#calcManifest()):
>
> // Copy old values into new manifest, when they
> // exist in the old one, but not in the new one
> merge(manifest, dot.getManifest());
>
> pax-logging-log4j2 bundle didn't have any Export-Package (as intended),
> so it just inherited it from org.apache.logging.log4j:log4j-core
> (definitely *not* as intended...).
>
> This is related to https://ops4j1.jira.com/browse/PAXLOGGING-240 and the
> discussion here[1].
>
> To be honest, the only thing I think is sensible here is to stop
> exporting anything from pax-logging-log4j2... Extensions should be done
> via fragments or pax-logging-api's org.ops4j.pax.logging.spi interfaces
> (like PaxAppender).
>
> WDYT?
>
> regards
> Grzegorz Grzybek
> ===
> [1]: https://groups.google.com/forum/#!msg/ops4j/yjqOzvrKRkc/t5BXmfyoBgAJ
>
> --
> --
> ------------------
> OPS4J - http://www.ops4j.org - op...@googlegroups.com
>
> ---
> You received this message because you are subscribed to the Google
> Groups "OPS4J" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to ops4j+un...@googlegroups.com
> <mailto:ops4j+un...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ops4j/CAAdXmhr1W2_3y56mpUUiErHFgKs13jFb2bd4CoC0rvcfaE9M8A%40mail.gmail.com
> <https://groups.google.com/d/msgid/ops4j/CAAdXmhr1W2_3y56mpUUiErHFgKs13jFb2bd4CoC0rvcfaE9M8A%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

Niclas Hedhman

unread,
May 23, 2019, 10:13:14 PM5/23/19
to OPS4J

Just because someone added a BND descriptor to Maven/Gradle, doesn't mean that the library is OSGi-capable, let alone OSGi-friendly. Please see https://articles.qos.ch/classloader.html for the starting point, and although the article discusses Java EE classloading, the OSGi scenario isn't any better for many libraries out there. Pax Logging made an attempt at overcoming the issues for all logging frameworks when running in OSGi, and it was a particularly meticulous beginning, when I ensured that nothing was left behind in the class space and that one should even be able to replace the pax-logging-service without stopping all the bundles, i.e. the whole app. Whether or not this goal has been maintained is unknown to me, I don't know if it stills hold true for the original implementation, and no idea if the log4j2 and logback impls have tried.

Almost all the 'embedding', and more importantly the replacements, have origins in the above.

Niclas

To unsubscribe from this group and stop receiving emails from it, send an email to ops4j+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ops4j/39649881-6014-cf4b-323c-dfcf045f35a5%40googlemail.com.

For more options, visit https://groups.google.com/d/optout.


--
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java

Grzegorz Grzybek

unread,
May 24, 2019, 12:52:01 AM5/24/19
to op...@googlegroups.com
Hello

Yes - in ideal world, every jar in Maven Central should be OSGi bundle and every closure of dependencies should be consistent.
Also, everyone forced to use OSGi hates it and switches to beautiful one-line µservices written in go-lang.
</sarcasm>

org.apache.logging.log4j:log4j-api and log4j-core (log4j2) are indeed OSGi bundles, but with own activators that do something not related to pax-logging.
Also nothing in Log4J2 implements org.osgi.service.log.LogService.

log4j-core does for example this:

private static void scanInstalledBundlesForPlugins(final BundleContext context) {
    final Bundle[] bundles = context.getBundles();
    for (final Bundle bundle : bundles) {
        // TODO: bundle state can change during this
        scanBundleForPlugins(bundle);
    }
}

As for pax-logging, Niclas - since last email I wrote >60 integration tests that check various assumptions you probably made when starting this project. There are tests for example that check how a piece of code using loggers from different frameworks/facades (JCL, JUL, Slf4J, JULI, Avalon, Log4j1 "api", ...) behaves when pax-logging-api or pax-logging-<backend> (or both) bundles are restarted/refreshed.
There are tests for extending log4j1/logback (log4j2 tests coming soon) via org.ops4j.pax.logging.spi interfaces or via fragments that just contain e.g., classes inheriting the Appender interface(s) (or filters, or error handlers, or layouts).

I'm getting more confident about pax-logging architecture and approach.

Soon we'll (Jean Baptiste Onofre is working on it) get R7 support (org.osgi.service.log 1.4).

Whether or not this goal has been maintained is unknown to me, I don't know if it stills hold true for the original implementation, and no idea if the log4j2 and logback impls have tried.

It holds. Both for log4j1 and logback and I'm just finishing to ensure that it's working for log4j2 too.
Private-Packaging without exporting of classes from "impl" jars (indeed it wasn't easy with log4j1) was and still is (IMO) good idea.

The fact that something is OSGi bundle, doesn't mean that original author took care of everything. For example log4j2 has "log4j-osgi" subproject, but it's simply:
 - org.apache.logging.log4j.osgi.tests.felix.FelixLoadApiBundleTest
 - org.apache.logging.log4j.osgi.tests.equinox.EquinoxLoadApiBundleTest

both these "tests" just install log4j-api, log4j-core, log4j-samples and log4j-1.2-api.

best regards
Grzegorz Grzybek

Niclas Hedhman

unread,
May 24, 2019, 2:48:23 AM5/24/19
to OPS4J

I have seen that you have done a lot of good work, and the only thing I can contribute nowadays is history/background and initial conditions/goals.


Cheers
Niclas


For more options, visit https://groups.google.com/d/optout.

Grzegorz Grzybek

unread,
May 24, 2019, 2:50:07 AM5/24/19
to op...@googlegroups.com
pt., 24 maj 2019 o 08:48 Niclas Hedhman <nic...@hedhman.org> napisał(a):

I have seen that you have done a lot of good work, and the only thing I can contribute nowadays is history/background and initial conditions/goals.

And I highly appreciate - those who don't know history are doomed to repeat it ;)

regards
Grzegorz

Christoph Läubrich

unread,
May 24, 2019, 4:36:58 AM5/24/19
to op...@googlegroups.com
But there is no need for starting these bundles. If you don't start them
the activator won't be executed but pax-logging can still use the
classes from them.
And if there are still issues, why not fix them in log4j instead of
providing an own repacked version?

I just wan't to point out that instead of embedding required classes for
pax-logging just importing the packages with proper version range would
be better. pax-logging can still provide the required service/spi on top
of it.

Am 24.05.19 um 06:51 schrieb Grzegorz Grzybek:
> Hello
>
> Yes - in ideal world, every jar in Maven Central should be OSGi bundle
> and every closure of dependencies should be consistent.
> Also, everyone forced to use OSGi hates it and switches to beautiful
> one-line µservices written in go-lang.
> </sarcasm>
>
> org.apache.logging.log4j:log4j-api and log4j-core (log4j2) are indeed
> OSGi bundles, but with own activators that do something not related to
> pax-logging.
> Also nothing in Log4J2 implements org.osgi.service.log.LogService.
>
> log4j-core does for example this:
>
> private static void scanInstalledBundlesForPlugins(final BundleContext
> context) {
>     final Bundle[] bundles = context.getBundles();
>     for (final Bundle bundle : bundles) {
> *        // TODO: bundle state can change during this*
> <mailto:nic...@hedhman.org>> napisał(a):
> <mailto:op...@googlegroups.com>
> >
> > ---
> > You received this message because you are subscribed to the
> Google
> > Groups "OPS4J" group.
> > To unsubscribe from this group and stop receiving emails from
> it, send
> > an email to ops4j+un...@googlegroups.com
> <mailto:ops4j%2Bunsu...@googlegroups.com>
> > <mailto:ops4j+un...@googlegroups.com
> <mailto:ops4j%2Bunsu...@googlegroups.com>>.
> <mailto:op...@googlegroups.com>
>
> ---
> You received this message because you are subscribed to the
> Google Groups "OPS4J" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to ops4j+un...@googlegroups.com
> <mailto:ops4j%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ops4j/39649881-6014-cf4b-323c-dfcf045f35a5%40googlemail.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Niclas Hedhman, Software Developer
> http://polygene.apache.org - New Energy for Java
>
> --
> --
> ------------------
> OPS4J - http://www.ops4j.org - op...@googlegroups.com
> <mailto:op...@googlegroups.com>
>
> ---
> You received this message because you are subscribed to the Google
> Groups "OPS4J" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to ops4j+un...@googlegroups.com
> <mailto:ops4j+un...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ops4j/CADmm%2BKf1FMysGrKFMM4DqTTKbsxx9skXDf8yZUPnF_tCMdYfUg%40mail.gmail.com
> <https://groups.google.com/d/msgid/ops4j/CADmm%2BKf1FMysGrKFMM4DqTTKbsxx9skXDf8yZUPnF_tCMdYfUg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.
>
> --
> --
> ------------------
> OPS4J - http://www.ops4j.org - op...@googlegroups.com
>
> ---
> You received this message because you are subscribed to the Google
> Groups "OPS4J" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to ops4j+un...@googlegroups.com
> <mailto:ops4j+un...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ops4j/CAAdXmhop9sFb8FP8L2cy_S8GeT5x72v51ouNAaOD5Wxbgk21Kw%40mail.gmail.com
> <https://groups.google.com/d/msgid/ops4j/CAAdXmhop9sFb8FP8L2cy_S8GeT5x72v51ouNAaOD5Wxbgk21Kw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Grzegorz Grzybek

unread,
May 24, 2019, 5:25:19 AM5/24/19
to op...@googlegroups.com
Hello

One of the reasons was that these pax-logging bundles usually start very early. In Karaf they're in "startup" stage (== etc/startup.properties), thus starting before even Features Service start.
That's how it worked since always and I didn't mean to change it.

regards
Grzegorz Grzybek

To unsubscribe from this group and stop receiving emails from it, send an email to ops4j+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ops4j/0b5bffa8-4254-f955-2529-9211e37276be%40googlemail.com.

Christoph Läubrich

unread,
May 24, 2019, 5:30:58 AM5/24/19
to op...@googlegroups.com
But you can still star them early, I don't see a reason why this should
not work...

Am 24.05.19 um 11:25 schrieb Grzegorz Grzybek:
> Hello
>
> One of the reasons was that these pax-logging bundles usually start very
> early. In Karaf they're in "startup" stage (== etc/startup.properties),
> thus starting before even Features Service start.
> That's how it worked since always and I didn't mean to change it.
>
> regards
> Grzegorz Grzybek
>
> pt., 24 maj 2019 o 10:36 'Christoph Läubrich' via OPS4J
> <op...@googlegroups.com <mailto:op...@googlegroups.com>> napisał(a):
> > <mailto:nic...@hedhman.org <mailto:nic...@hedhman.org>>> napisał(a):
> >         <mailto:op...@googlegroups.com
> <mailto:op...@googlegroups.com>>
> >          >
> >          > ---
> >          > You received this message because you are subscribed
> to the
> >         Google
> >          > Groups "OPS4J" group.
> >          > To unsubscribe from this group and stop receiving
> emails from
> >         it, send
> >          > an email to ops4j+un...@googlegroups.com
> <mailto:ops4j%2Bunsu...@googlegroups.com>
> >         <mailto:ops4j%2Bunsu...@googlegroups.com
> <mailto:ops4j%252Buns...@googlegroups.com>>
> >          > <mailto:ops4j+un...@googlegroups.com
> <mailto:ops4j%2Bunsu...@googlegroups.com>
> >         <mailto:ops4j%2Bunsu...@googlegroups.com
> <mailto:ops4j%252Buns...@googlegroups.com>>>.
> >          > To view this discussion on the web visit
> >          >
> >
> https://groups.google.com/d/msgid/ops4j/CAAdXmhr1W2_3y56mpUUiErHFgKs13jFb2bd4CoC0rvcfaE9M8A%40mail.gmail.com
> >
> >          >
> >
>  <https://groups.google.com/d/msgid/ops4j/CAAdXmhr1W2_3y56mpUUiErHFgKs13jFb2bd4CoC0rvcfaE9M8A%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> >          > For more options, visit
> https://groups.google.com/d/optout.
> >
> >         --
> >         --
> >         ------------------
> >         OPS4J - http://www.ops4j.org - op...@googlegroups.com
> <mailto:op...@googlegroups.com>
> >         <mailto:op...@googlegroups.com
> <mailto:op...@googlegroups.com>>
> >
> >         ---
> >         You received this message because you are subscribed to the
> >         Google Groups "OPS4J" group.
> >         To unsubscribe from this group and stop receiving emails from
> >         it, send an email to ops4j+un...@googlegroups.com
> <mailto:ops4j%2Bunsu...@googlegroups.com>
> >         <mailto:ops4j%2Bunsu...@googlegroups.com
> <mailto:ops4j%252Buns...@googlegroups.com>>.
> >         To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/ops4j/39649881-6014-cf4b-323c-dfcf045f35a5%40googlemail.com.
> >         For more options, visit https://groups.google.com/d/optout.
> >
> >
> >
> >     --
> >     Niclas Hedhman, Software Developer
> > http://polygene.apache.org - New Energy for Java
> >
> >     --
> >     --
> >     ------------------
> >     OPS4J - http://www.ops4j.org - op...@googlegroups.com
> <mailto:op...@googlegroups.com>
> >     <mailto:op...@googlegroups.com <mailto:op...@googlegroups.com>>
> >
> >     ---
> >     You received this message because you are subscribed to the
> Google
> >     Groups "OPS4J" group.
> >     To unsubscribe from this group and stop receiving emails from it,
> >     send an email to ops4j+un...@googlegroups.com
> <mailto:ops4j%2Bunsu...@googlegroups.com>
> >     <mailto:ops4j+un...@googlegroups.com
> <mailto:ops4j%2Bunsu...@googlegroups.com>>.
> >     To view this discussion on the web visit
> >
> <mailto:op...@googlegroups.com>
> >
> > ---
> > You received this message because you are subscribed to the Google
> > Groups "OPS4J" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to ops4j+un...@googlegroups.com
> <mailto:ops4j%2Bunsu...@googlegroups.com>
> > <mailto:ops4j+un...@googlegroups.com
> <mailto:ops4j%2Bunsu...@googlegroups.com>>.
> > To view this discussion on the web visit
> >
> <mailto:op...@googlegroups.com>
>
> ---
> You received this message because you are subscribed to the Google
> Groups "OPS4J" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to ops4j+un...@googlegroups.com
> <mailto:ops4j%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> --
> --
> ------------------
> OPS4J - http://www.ops4j.org - op...@googlegroups.com
>
> ---
> You received this message because you are subscribed to the Google
> Groups "OPS4J" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to ops4j+un...@googlegroups.com
> <mailto:ops4j+un...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ops4j/CAAdXmhpA%2B2TW4zKZ_TWb_5S3GPM-cf%2BNu04sG5a8FxCdG4EY%3DQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/ops4j/CAAdXmhpA%2B2TW4zKZ_TWb_5S3GPM-cf%2BNu04sG5a8FxCdG4EY%3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Niclas Hedhman

unread,
May 25, 2019, 8:08:00 AM5/25/19
to op...@googlegroups.com

Trust me, I tried to make it work with the deliverables from Log4j v1 and all of the APIs, and it was simply not possible, IF you maintain the goal that reloading/replacing the logging service implementation should be possible without stopping the entire application and without having memory leaks in the class space.

IF you don't care about these goals, then by all means go ahead and use some logging bundles without Pax Logging.

IF you are so certain that it can be done in Pax Logging without giving up these goals, then please help those of us who have tried and failed. Show me the code.... because the devil is in the details.



Cheers
Niclas

To unsubscribe from this group and stop receiving emails from it, send an email to ops4j+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ops4j/8d8abdfd-5469-b96c-9aa7-9cd64517280e%40googlemail.com.
Reply all
Reply to author
Forward
0 new messages