winstone Jetty java 11 release strategy

[Olivier Lamy]

Hi
As now we can bump from Java 8 to Java 11, I'd like to start release Winstone to include more recent releases of Jetty (e.g 10).

I prefer asking here before starting it. Rather than having to ignore complaints once it's started :)

The goal is to keep maintaining a 5.x branch until Java 11 reaches Jenkins core LTS.

Make master branch 6.x (java 11 and Jetty 10) and release it.

Thoughts? 

cheers

--

Olivier

[Olivier Lamy]

Oh sadly the parent bump has been merged https://github.com/jenkinsci/winstone/pull/228

before the last Jetty 9.4.x bump.

Seriously guys, it's a community project!! so you could wait/listen to comments before merging stuff..

And by the way, discuss changes...

--

Olivier Lamy
http://twitter.com/olamy | http://linkedin.com/in/olamy

[Basil Crow]

On Wed, Jun 22, 2022 at 6:33 PM Olivier Lamy <olive...@gmail.com> wrote:
>
> As now we can bump from Java 8 to Java 11, I'd like to start release Winstone to include more recent releases of Jetty (e.g 10).

JENKINS-68694 FTR

> The goal is to keep maintaining a 5.x branch until Java 11 reaches Jenkins core LTS.
> Make master branch 6.x (java 11 and Jetty 10) and release it.

+1. The desired outcome can be achieved as follows: I can set up a 5.x
branch based on 5.26 (i.e., parent POM 1.76), upgrade Jetty from
9.4.46.v20220331 to 9.4.48.v20220622, and release Winstone 5.27 from
that branch (setting the version to 6.0-SNAPSHOT on the main branch
once that is done). Then the implementation of JENKINS-68694 can be
merged to the main branch (which is already on parent POM 1.77) and
6.0 can be released from the main branch based on Jetty 10.

[Olivier Lamy]

As I have started the work I'm happy to finish. So my email to propose a plan before doing anything.

Seriously before doing some big changes please communicate, wait for others to review PRs.....

We are supposed to be a community working on a common project.....

 

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFwNDjo%3DZgxcqWPs3CsqtEkGNOUNdvW4oUHaO%3Di-ENjvXv%3DA_Q%40mail.gmail.com.

[Basil Crow]

On Wed, Jun 22, 2022 at 7:09 PM Olivier Lamy <olive...@gmail.com> wrote:
>
> As I have started the work I'm happy to finish.

Great! I saw that you successfully created a 5.x branch. Do not forget
to set the revision to 6.0 on the main branch.

[Basil Crow]

Of note is that Jetty 10 has grown a dependency on SLF4J. At first I
was concerned that this might conflict with the version of SLF4J we
ship for plugins, but after reading
https://www.eclipse.org/jetty/documentation/jetty-10/programming-guide/index.html#pg-server-http-handler-use-webapp-context
I managed to convince myself that Jetty could load its own copy of
SLF4J in one class loader and Jenkins core could load its own copy of
SLF4J in a different class loader and both would delegate to the same
copy of JUL seamlessly, in theory. The reasoning behind this is that
Jetty 10 marks all Jetty classes as "server classes" which would in
theory hide them (and all classes loaded by them, including SLF4J)
from the web application (note that this definition of "server class"
is different from the one in Jetty 9!). So there is no possibility of
Jenkins core or a Jenkins plugin being able to successfully load a
Jetty class, and therefore no possibility of Jenkins core or a Jenkins
plugin being able to see the Jetty copy of SLF4J rather than the
Jenkins core copy.

But this chain of reasoning led me to another conclusion as well: if
we can hide SLF4J, then why not also hide Bouncy Castle in similar
fashion? (Context: Bouncy Castle is the only obvious way to implement
the --httpsPrivateKey and --httpsCertificate options for Java 17,
since the Java Platform removed access to
"sun.security.util.DerInputStream".) In theory we could configure
Bouncy Castle classes to be "server classes", which would allow Jetty
to see them but not Jenkins core and plugins, much like the Jetty
classes themselves (which are "server classes" in Jetty 10) are not
visible to web applications. This would in theory allow a Bouncy
Castle based implementation of --httpsPrivateKey and
--httpsCertificate.

We may want to remove --httpsPrivateKey and --httpsCertificate
regardless, because they have been deprecated since 2016. I certainly
have no interest in prototyping the above idea myself. But I did want
to note that it seems like a plausible option. I guess the question
is, does anyone care enough about --httpsPrivateKey and
--httpsCertificate to try and prototype the above solution, or do we
all agree that it is time to dispense with these options that have
been deprecated since 2016?

[Olivier Lamy]

ETA 

release 5.27 (e.g Jettty 9.4.48) done.

There is a branch called jetty-10 (PR https://github.com/jenkinsci/winstone/pull/181) including upgrade to Jetty 10.0.11

Personally (and conservative POV) I do not feel a rush to push release this and include it ASAP in weekly.

but cool kids may have different opinions about doing it ASAP :) well do it.

As weekly already has big changes coming with java 11.

Probably not many issues but better to not have collisions with possible issues coming with a Jetty upgrade as well. 

Maybe incremental build could be tested with all the chain ath, pct etc...

I will be off until next weekend so if you need changes feel free to push them in the branch. It's supposed to be a community branch to share the work.

Or if someone wants to take over this feel free to do so. 

I just feel I need to step away from participating in this.

Cheers and enjoy your weekend,

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFwNDjqFgoauGsit9xq%3D0bML%3DQ%2B_Ln%2BoKhBDXF3dUr0E8Zq2EQ%40mail.gmail.com.

[Basil Crow]

On Thu, Jun 23, 2022 at 7:54 PM Olivier Lamy <olive...@gmail.com> wrote:
> Personally (and conservative POV) I do not feel a rush to push release this and include it ASAP in weekly.
> but cool kids may have different opinions about doing it ASAP :) well do it.
> As weekly already has big changes coming with java 11.
> Probably not many issues but better to not have collisions with possible issues coming with a Jetty upgrade as well.

I could see arguments for both sides. On the one hand, the content of
the September LTS is heavily focused on platform improvements, and
this fits with that theme. On the other hand, it is an orthogonal
change, as you note, and it may be tough to differentiate the cause of
an issue when it could be due to either the Java 11 upgrade or the
Jetty upgrade. I actually had a similar concern with Guava and Guice
(and with Ant changes prior to that), so I employed a strategy of
staggering changes across multiple weeklies. That way, if a user
encounters a problem, we can ask them to roll back to an intermediate
weekly and thereby bisect more easily. So I think this concern could
be alleviated by waiting a week or two after the Java 11 weekly lands
before we have a Jetty 10 weekly. The staggering effect should
hopefully make it easier to support users via bisection. And I think
waiting a week or two still gives us plenty of soak time before LTS.
In fact, it may even take us about that long to test the Jetty 10 PR
with PCT/ATH anyway.

(BTW, this isn't the only change I am planning to stagger in this way.
I am also holding off on JUnit 5 for the same reason: trying to
stagger risky changes across multiple weeklies for ease of bisection.)

> Maybe incremental build could be tested with all the chain ath, pct etc...

I concur. PCT/ATH is clearly appropriate.

> I will be off until next weekend so if you need changes feel free to push them in the branch. It's supposed to be a community branch to share the work.

You have definitely done the lion's share of the work in that branch,
for which I am exceedingly grateful. I did not realize you were OK
with others pushing to a communal branch, but I am delighted that you
are! I love working this way, and perhaps we can use that as a pattern
for collaboration going forward. I just pushed two more commits to the
branch, and we now have a green PR build! We should be able to start
testing that build with the core test suite and PCT/ATH after the next
weekly release. If nobody volunteers to do it I will try and run some
tests after I finish dealing with the Java 11 tasks.

> I will be off until next weekend

> Cheers and enjoy your weekend,

I hope you enjoy the weekend as well, Olivier. I look forward to
finishing this up together with you when you return, if it isn't
finished already! As you wrote many times, this is all about coming
together as a community, and I am excited we are so close to the
finish line.

[Basil Crow]

On Thu, Jun 23, 2022 at 9:25 PM Basil Crow <m...@basilcrow.com> wrote:
> We should be able to start
> testing that build with the core test suite and PCT/ATH after the next
> weekly release. If nobody volunteers to do it I will try and run some
> tests after I finish dealing with the Java 11 tasks.

I ran the core test suite and BOM tests on tip-of-trunk Winstone
(based on Jetty 10.0.11) with no issues, and I did some interactive
testing as well with no issues. I plan to release Winstone 6.0 within
the next day or two toward integration in one of the next few
weeklies, possibly 2.358 if no other issues are identified.

[jn...@cloudbees.com]

>  But this chain of reasoning led me to another conclusion as well: if
we can hide SLF4J, then why not also hide Bouncy Castle in similar
fashion? (Context: Bouncy Castle is the only obvious way to implement
the --httpsPrivateKey and --httpsCertificate options for Java 17,
since the Java Platform removed access to
"sun.security.util.DerInputStream".) In theory we could configure
Bouncy Castle classes to be "server classes", which would allow Jetty
to see them but not Jenkins core and plugins, much like the Jetty
classes themselves (which are "server classes" in Jetty 10) are not
visible to web applications. This would in theory allow a Bouncy
Castle based implementation of 

One reason not to do that would be because it embeds (more) cryto into the project, which causes issues for FIPS compliance.

Do we need to take the key and certificate separately, winstone can load from a keystore - and PKCS12 is supported.  (it has been removed anyway in https://github.com/jenkinsci/winstone/pull/232 )

/James