[VOTE] 3.8.0 RC3

[Josep Prat]

Hello Kafka users, developers and client-developers,

This is the fourth candidate for release of Apache Kafka 3.8.0.

Some of the major features included in this release are:

* KIP-1028: Docker Official Image for Apache Kafka

* KIP-974: Docker Image for GraalVM based Native Kafka Broker

* KIP-1036: Extend RecordDeserializationException exception

* KIP-1019: Expose method to determine Metric Measurability

* KIP-1004: Enforce tasks.max property in Kafka Connect

* KIP-989: Improved StateStore Iterator metrics for detecting leaks

* KIP-993: Allow restricting files accessed by File and Directory ConfigProviders

* KIP-924: customizable task assignment for Streams

* KIP-813: Shareable State Stores

* KIP-719: Deprecate Log4J Appender

* KIP-390: Support Compression Level

* KIP-1018: Introduce max remote fetch timeout config for DelayedRemoteFetch requests

* KIP-1037: Allow WriteTxnMarkers API with Alter Cluster Permission

* KIP-1047 Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder

* KIP-899: Allow producer and consumer clients to rebootstrap

Release notes for the 3.8.0 release:
https://home.apache.org/~jlprat/kafka-3.8.0-rc3/RELEASE_NOTES.html

*** Please download, test and vote by Friday, July 26, 9am PT


Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
https://home.apache.org/~jlprat/kafka-3.8.0-rc3/

* Docker release artifact to be voted upon:
apache/kafka:3.8.0-rc3
apache/kafka-native:3.8.0-rc3

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/

* Javadoc:
https://home.apache.org/~jlprat/kafka-3.8.0-rc3/javadoc/

* Tag to be voted upon (off 3.8 branch) is the 3.8.0 tag:
https://github.com/apache/kafka/releases/tag/3.8.0-rc3

* Documentation:
https://kafka.apache.org/38/documentation.html

* Protocol:
https://kafka.apache.org/38/protocol.html

* Successful Jenkins builds for the 3.8 branch:
Unit/integration tests: https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.8/73/
System tests: Still running

* Successful Docker Image Github Actions Pipeline for 3.8 branch:
Docker Build Test Pipeline (JVM): https://github.com/apache/kafka/actions/runs/10055827182
Docker Build Test Pipeline (Native): https://github.com/apache/kafka/actions/runs/10055829295


Thanks,

--

Josep Prat Open Source Engineering Director, Aiven josep...@aiven.io   |   +491715557497 aiven.io   |

Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B

[Josep Prat]

Here is the link to the system tests:

https://confluent-open-source-kafka-system-test-results.s3-us-west-2.amazonaws.com/3.8/2024-07-22--001.ffbb03b2-61f4-4ebb-ae1f-af5c753682fb--1721733000--confluentinc--3.8--9a2b34b68c/report.html

The Quota tests are known to fail in this CI system. Regarding the other tests, they run successfully in the past and they are now timeouting.

Best,

[Chris Egerton]

Forwarding my response to the other mailing list threads; apologies for missing the reply-all the first time!

---------- Forwarded message ---------
From: Chris Egerton <chr...@aiven.io>
Date: Tue, Jul 23, 2024 at 3:45 PM
Subject: Re: [VOTE] 3.8.0 RC3
To: <us...@kafka.apache.org>

Hi Josep,

Thanks for running this release! I'm +1 (binding).


To verify, I:
- Built from source using Java 11 with both:
- - the 3.8.0-rc3 tag on GitHub
- - the source artifact from https://home.apache.org/~jlprat/kafka-3.8.0-rc3/kafka-3.8.0-src.tgz
- Checked signatures and checksums
- Ran the quickstart using both:
- - The build artifact from https://home.apache.org/~jlprat/kafka-3.8.0-rc3/kafka_2.13-3.8.0.tgz with Java 11 and Scala 13 in KRaft mode
- - Our JVM-based broker Docker image, apache/kafka:3.8.0-rc3
- Ran all unit tests
- Ran all integration tests for Connect and MM2

A few small, non-blocking notes:

1) The release notes categorize KAFKA-16445 [1] as an improvement, but I believe it should be listed as a new feature instead.

2) The following unit tests failed the first time around, but passed when run a second time:

- (clients) SaslAuthenticatorTest.testMissingUsernameSaslPlain()

- (core) ProducerIdManagerTest.testUnrecoverableErrors(UNKNOWN_SERVER_ERROR)

- (core) RemoteLogManagerTest.testCopyQuota(false)

- (core) SocketServerTest.testClientDisconnectionWithOutstandingReceivesProcessedUntilFailedSend()

- (core) ZkMigrationIntegrationTest.testMigrateTopicDeletions [7] Type=ZK, MetadataVersion=3.7-IV4,Security=PLAINTEXT

  - This is also not actually a unit test, but an integration test. Looks like we haven't classified it correctly?

[1] - https://issues.apache.org/jira/browse/KAFKA-16445

Cheers,

Chris

> **** Please download, test and vote by Friday, July 26, 9am PT*

> [image: Aiven] <https://www.aiven.io>
>
> *Josep Prat*
> Open Source Engineering Director, *Aiven*
> josep...@aiven.io   |   +491715557497
> aiven.io <https://www.aiven.io>   |
> <https://www.facebook.com/aivencloud>
> <https://www.linkedin.com/company/aiven/>   <https://twitter.com/aiven_io>
> *Aiven Deutschland GmbH*

> Alexanderufer 3-7, 10117 Berlin
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> Amtsgericht Charlottenburg, HRB 209739 B
>


--

[image: Aiven] <https://www.aiven.io>

*Josep Prat*
Open Source Engineering Director, *Aiven*
josep...@aiven.io   |   +491715557497
aiven.io <https://www.aiven.io>   |   <https://www.facebook.com/aivencloud>
  <https://www.linkedin.com/company/aiven/>   <https://twitter.com/aiven_io>
*Aiven Deutschland GmbH*

[Josep Prat]

Thank you all who reviewed this release candidate and the previous ones.

This vote passes with 7 +1 votes (3 bindings) and no 0 or -1 votes.

+1 votes
PMC Members:
* Chris Egerton

* Greg Harris

* Mickael Maison

Committers:
* Igor Soarez

Community:

* Murali Basani

* Jakub Scholz

* Krishna Agarwal

0 votes
* No votes

-1 votes
* No votes

I'll continue with the release process and the release announcement will follow in the next few days.

Best,

On Thu, Jul 25, 2024 at 7:44 PM Krishna Agarwal <krishna06...@gmail.com> wrote:

Hi Josep,

Thanks for the release candidate.

+1 (non-binding)

I tested and verified the docker image artifact
apache/kafka-native:3.8.0-rc3
- verified create topic, produce messages and consume messages flow.
- verified the html documentation for docker image.

I also ran the System tests by bringing up Kafka in the native mode.
Following tests failed in the run.

-
> tests/kafkatest/tests/tools/replica_verification_test.py::ReplicaVerificationToolTest.test_replica_lags@
> {"metadata_quorum":"ISOLATED_KRAFT"}
> -
> tests/kafkatest/tests/tools/replica_verification_test.py::ReplicaVerificationToolTest.test_replica_lags@
> {"metadata_quorum":"ZK"}
> - tests/kafkatest/tests/client/quota_test.py::QuotaTest.test_quota@
> {"quota_type":"client-id","old_client_throttling_behavior":true}
> - tests/kafkatest/tests/client/quota_test.py::QuotaTest.test_quota@
> {"quota_type":"(user,client-id)","override_quota":true}
> - tests/kafkatest/tests/client/quota_test.py::QuotaTest.test_quota@
> {"quota_type":"client-id","old_broker_throttling_behavior":true}
> - tests/kafkatest/tests/client/quota_test.py::QuotaTest.test_quota@
> {"quota_type":"client-id","consumer_num":2}
> - tests/kafkatest/tests/client/quota_test.py::QuotaTest.test_quota@
> {"quota_type":"(user,client-id)","override_quota":false}
> - tests/kafkatest/tests/client/quota_test.py::QuotaTest.test_quota@
> {"quota_type":"client-id","override_quota":false}
> - tests/kafkatest/tests/client/quota_test.py::QuotaTest.test_quota@
> {"quota_type":"user","override_quota":true}
> - tests/kafkatest/tests/client/quota_test.py::QuotaTest.test_quota@
> {"quota_type":"client-id","override_quota":true}
> - tests/kafkatest/tests/client/quota_test.py::QuotaTest.test_quota@
> {"quota_type":"user","override_quota":false}
> -
> tests/kafkatest/tests/connect/connect_distributed_test.py::ConnectDistributedTest.test_dynamic_logging
> -
> tests/kafkatest/tests/connect/connect_distributed_test.py::ConnectDistributedTest.test_pause_and_resume_sink@
> {"connect_protocol":"sessioned","metadata_quorum":"ISOLATED_KRAFT","use_new_coordinator":true,"group_protocol":"consumer"}
>

On re-running the failed tests locally, *all tests passed except*
`tests/kafkatest/tests/tools/replica_verification_test.py::ReplicaVerificationToolTest.test_replica_lags@{"metadata_quorum":"ZK"}`(timeout
error).

All looks good for this docker image artifact!

Regards,
Krishna

On Wed, Jul 24, 2024 at 10:46 PM Mickael Maison <mickael...@gmail.com>
wrote:

> Hi Josep,
>
> I have:
> - checked signatures and checksums
> - run ZK and KRaft quickstarts with the Scala 2.13 binaries and with
> both the kafka and kafka-native images
> - built and run tests from source with Java 17
>
> +1 (binding)
>
> Thanks,
> Mickael
>
> On Wed, Jul 24, 2024 at 4:22 PM Igor Soarez <soa...@apache.org> wrote:
> >
> > Hi Josep,
> >
> > That makes sense to me, thanks for clarifying.
> >
> > +1 non-binding from me then.
> >
> > --
> > Igor
>

--

Josep Prat Open Source Engineering Director, Aiven josep...@aiven.io   |   +491715557497 aiven.io   |

Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin

Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, Anna Richardson, Kenneth Chen