diff --git a/.github/workflows/build_test_pr.yml b/.github/workflows/build_test_pr.yml
index 3c31914..260ace2 100644
--- a/.github/workflows/build_test_pr.yml
+++ b/.github/workflows/build_test_pr.yml
@@ -60,7 +60,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Upload
- uses: actions/upload-artifact@v2
+ uses: actions/upload-artifact@v4
with:
name: Event File
path: ${{ github.event_path }}
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9873af7..c04a527 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,11 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/)
and this project adheres to [Semantic Versioning](http://semver.org/).
+## [[2.1.0]] (https://github.com/AppsFlyer/ketu/pull/22) - 2025-09-08
+### Changed
+- Java clients library `org.apache.kafka/kafka-clients` upgraded from [3.3.1](https://kafka.apache.org/33/documentation.html) to [3.9.1](https://kafka.apache.org/39/documentation.html)
+- Kafka clients changes analysis is available [here](kafka-client-changes-analysis.md)
+
## [[2.0.0]] (https://github.com/AppsFlyer/ketu/pull/19) - 2024-09-03
### Changed
- consumer decorator API breaking change - use ConsumerDecorator protocol instead of `consumer-decorator` function.
diff --git a/README.md b/README.md
index 4313097..d844ca4 100644
--- a/README.md
+++ b/README.md
@@ -8,7 +8,7 @@
A Clojure Apache Kafka client with core.async api
```clojure
-[com.appsflyer/ketu "2.0.0"]
+[com.appsflyer/ketu "2.1.0"]
```
## Features
@@ -62,7 +62,7 @@ Consume a name string from kafka and produce a greeting string for that name bac
Anything that is not documented is not supported and might change.
Read more about the default values used by the underlying Kafka clients
-v3.3.1 [here](https://kafka.apache.org/33/documentation.html)
+v3.9.1 [here](https://kafka.apache.org/39/documentation.html)
Note: `int` is used for brevity but can also mean `long`. Don't worry about it.
@@ -225,6 +225,11 @@ The decorator processes all immediately available commands in the commands-chan,
(source/stop! source))))
```
+## Java Kafka client versions
+- `ketu` version 1.0.0+ uses `org.apache.kafka/kafka-clients` version 3.3.1
+- `ketu` version 2.1.0+ uses `org.apache.kafka/kafka-clients` version 3.9.1
+ - For a comprehensive list of changes in the java client, see [here](kafka-client-changes-analysis.md)
+
## Development & Contribution
We welcome feedback and would love to hear about use-cases other than ours. You can open issues, send pull requests,
diff --git a/kafka-client-changes-analysis.md b/kafka-client-changes-analysis.md
new file mode 100644
index 0000000..1edafb9
--- /dev/null
+++ b/kafka-client-changes-analysis.md
@@ -0,0 +1,193 @@
+# Apache Kafka Client Library Changes Analysis
+
+## Version 3.3.1 to 3.9.1
+
+**Analysis Date:** September 2025
+**Library:** org.apache.kafka/kafka-clients
+**Version Range:** 3.3.1 → 3.9.1
+
+---
+
+## Executive Summary
+
+This document provides a comprehensive analysis of all changes between Apache Kafka client library versions 3.3.1 and
+3.9.1, with particular focus on breaking changes, configuration modifications, and default behavior changes. The
+analysis is organized with breaking changes listed first, followed by configuration changes and security updates.
+
+**Important Note:** Not all assumed intermediate versions (3.4.0, 3.5.0, 3.6.0, etc.) may have been released as major
+versions. The analysis focuses on documented changes across the version range.
+
+---
+
+## BREAKING CHANGES
+
+| Version | Change Type | Configuration/Component | Description | Impact | Source |
+|-----------|------------------------------|-------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------|
+| **3.9.1** | **Security Breaking Change** | `sasl.jaas.config` | Disabled `com.sun.security.auth.module.JndiLoginModule` and `com.sun.security.auth.module.LdapLoginModule` by default | Users must explicitly allow these modules if needed | [Apache Kafka CVE List](https://kafka.apache.org/cve-list.html) |
+| **3.9.1** | **Security Breaking Change** | `sasl.oauthbearer.token.endpoint.url`
`sasl.oauthbearer.jwks.endpoint.url` | Introduced `org.apache.kafka.sasl.oauthbearer.allowed.urls` system property | In 3.9.1, all URLs accepted by default; in 4.0.0+, empty list by default | [Apache Kafka CVE List](https://kafka.apache.org/cve-list.html) |
+| **3.9.0** | **Deprecation** | `delete-config` option in `kafka-topics.sh` | Deprecated `delete-config` option | Users should use `--alter --delete-config` with `kafka-configs.sh` or Admin API | [Confluent Platform Release Notes](https://docs.confluent.io/platform/current/release-notes/index.html) |
+| **3.9.0** | **Deprecation** | `offsets.commit.required.acks` | Deprecated configuration | Will be removed in Kafka 4.0 | [Apache Kafka Blog](https://kafka.apache.org/blog) |
+| **3.9.0** | **Deprecation** | Log4J Appender | Deprecated Log4J Appender | Expected to be removed in Kafka 4.0 | [Apache Kafka Blog](https://kafka.apache.org/blog) |
+| **3.9.0** | **Deprecation** | `kafka.serializer.Decoder` | Deprecated `kafka.serializer.Decoder` | Replaced by `org.apache.kafka.tools.api.Decoder` | [Apache Kafka Blog](https://kafka.apache.org/blog) |
+| **3.8.0** | **Deprecation** | `offsets.commit.required.acks` | Deprecated configuration | Will be removed in Kafka 4.0 | [Apache Kafka Blog](https://kafka.apache.org/blog) |
+| **3.8.0** | **Deprecation** | Log4J Appender | Deprecated Log4J Appender | Expected to be removed in Kafka 4.0 | [Apache Kafka Blog](https://kafka.apache.org/blog) |
+| **3.8.0** | **Deprecation** | `kafka.serializer.Decoder` | Deprecated `kafka.serializer.Decoder` | Replaced by `org.apache.kafka.tools.api.Decoder` | [Apache Kafka Blog](https://kafka.apache.org/blog) |
+| **3.7.0** | **Deprecation** | Client APIs prior to 2.1 | Client APIs released before Kafka 2.1 marked as deprecated | Will be removed in Kafka 4.0 | [Apache Kafka 3.7.0 Release Announcement](https://kafka.apache.org/blog) |
+| **3.7.0** | **Deprecation** | Java 11 Support | Java 11 support for Kafka broker deprecated | Planned for removal in Kafka 4.0 | [Apache Kafka 3.7.0 Release Announcement](https://kafka.apache.org/blog) |
+| **3.5.0** | **Deprecation** | ZooKeeper | ZooKeeper marked as deprecated | Planned for removal in Kafka 4.0 | [Apache Kafka Blog](https://kafka.apache.org/blog) |
+
+---
+
+## CONFIGURATION CHANGES AND DEFAULT BEHAVIOR MODIFICATIONS
+
+| Version | Configuration Parameter | Change Type | Description | Impact | Source |
+|-----------|-------------------------------------------------------------------------------|--------------------------|----------------------------------------------------------------------|-----------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------|
+| **3.9.1** | `org.apache.kafka.disallowed.login.modules` | **New System Property** | Added system property to disable specific login modules in SASL JAAS | Enhances security by preventing use of vulnerable modules | [Apache Kafka CVE List](https://kafka.apache.org/cve-list.html) |
+| **3.9.1** | `sasl.jaas.config` | **Security Fix** | Fixed deserialization of untrusted data vulnerability | Users should upgrade to 3.9.1+ and review configurations | [Snyk Vulnerability Report](https://security.snyk.io/package/maven/org.apache.kafka%3Akafka-clients/3.9.0) |
+| **3.9.1** | `sasl.oauthbearer.token.endpoint.url`
`sasl.oauthbearer.jwks.endpoint.url` | **Security Fix** | Fixed Server-Side Request Forgery (SSRF) vulnerability | Users should validate and restrict these configurations | [Snyk Vulnerability Report](https://security.snyk.io/package/maven/org.apache.kafka%3Akafka-clients/3.9.0) |
+| **3.9.1** | Java 23 Support | **New Feature** | Added support for Java 23 | Applications must ensure compatibility with Java 23 | [Apache Kafka 3.9.1 Release Announcement](https://kafkacommunity.blogspot.com/2025/05/announce-apache-kafka-391.html) |
+| **3.8.0** | `remote.fetch.max.wait.ms` | **New Configuration** | New timeout parameter for delayed remote fetch requests | Allows users to configure timeout based on workload | [Apache Kafka Blog](https://kafka.apache.org/blog) |
+| **3.7.0** | `org.apache.kafka.automatic.config.providers` | **New System Property** | System property to disable automatic config providers | Setting to `none` can mitigate security vulnerabilities | [IBM Security Bulletin](https://www.ibm.com/support/pages/security-bulletin-kafka-client-library-upgraded-kafka-clients-391-1) |
+| **3.7.0** | `ConfigProviders` interface | **Security Fix** | Addressed vulnerability allowing reading arbitrary disk contents | Users should upgrade and configure appropriate allowlists | [Snyk Vulnerability Report](https://security.snyk.io/package/maven/org.apache.kafka%3Akafka-clients/3.1.1) |
+| **3.3.0** | `replica.lag.max.messages` | **Removed** | Parameter removed | Partition leaders no longer consider lagging message count | [Confluent Platform 3.3.0 Release Notes](https://docs.confluent.io/legacy/platform/3.3.0/release-notes.html) |
+| **3.3.0** | `replica.lag.time.max.ms` | **Behavior Change** | Now refers to time since replica last caught up | Replicas not caught up within time are considered out of sync | [Confluent Platform 3.3.0 Release Notes](https://docs.confluent.io/legacy/platform/3.3.0/release-notes.html) |
+| **3.3.0** | `reserved.broker.max.id` | **Default Value Change** | Broker IDs above 1000 now reserved by default | If existing broker IDs exceed threshold, increase configuration | [Confluent Platform 3.3.0 Release Notes](https://docs.confluent.io/legacy/platform/3.3.0/release-notes.html) |
+
+---
+
+## SECURITY VULNERABILITIES ADDRESSED
+
+| Version | CVE/Security Issue | Configuration Affected | Description | Source |
+|-----------|-------------------------------|-------------------------------------------------------------------------------|------------------------------------------------|------------------------------------------------------------------------------------------------------------|
+| **3.9.1** | Deserialization Vulnerability | `sasl.jaas.config` | Fixed improper handling of configuration data | [Snyk Vulnerability Report](https://security.snyk.io/package/maven/org.apache.kafka%3Akafka-clients/3.9.0) |
+| **3.9.1** | SSRF Vulnerability | `sasl.oauthbearer.token.endpoint.url`
`sasl.oauthbearer.jwks.endpoint.url` | Fixed improper handling of URL configurations | [Snyk Vulnerability Report](https://security.snyk.io/package/maven/org.apache.kafka%3Akafka-clients/3.9.0) |
+| **3.7.0** | ConfigProvider Vulnerability | `ConfigProviders` interface | Fixed unauthorized access to files/directories | [Snyk Vulnerability Report](https://security.snyk.io/package/maven/org.apache.kafka%3Akafka-clients/3.1.1) |
+
+---
+
+## DETAILED BREAKING CHANGES ANALYSIS
+
+### 1. SASL JAAS Configuration Changes (3.9.1)
+
+**Impact:** HIGH - Security-related breaking change
+
+- **What Changed:** Default behavior now disables `JndiLoginModule` and `LdapLoginModule`
+- **Why:** Security vulnerability mitigation
+- **Action Required:** If using these modules, explicitly configure them or use alternative authentication methods
+
+### 2. OAuth Bearer URL Restrictions (3.9.1)
+
+**Impact:** MEDIUM - Configuration change with future breaking change
+
+- **What Changed:** New system property `org.apache.kafka.sasl.oauthbearer.allowed.urls`
+- **Why:** Prevent SSRF attacks
+- **Action Required:** In Kafka 4.0+, URLs must be explicitly allowed (empty list by default)
+
+### 3. ZooKeeper Deprecation (3.5.0)
+
+**Impact:** HIGH - Major architectural change
+
+- **What Changed:** ZooKeeper marked as deprecated
+- **Why:** Migration to KRaft mode for better performance and simplicity
+- **Action Required:** Plan migration to KRaft mode before Kafka 4.0
+
+### 4. Java 11 Deprecation (3.7.0)
+
+**Impact:** MEDIUM - Runtime environment change
+
+- **What Changed:** Java 11 support deprecated for Kafka broker
+- **Why:** Focus on newer Java versions
+- **Action Required:** Upgrade to Java 17+ for future compatibility
+
+---
+
+## MIGRATION RECOMMENDATIONS
+
+### Immediate Actions (Version 3.9.1)
+
+1. **Upgrade to 3.9.1** for critical security fixes
+2. **Review SASL configurations** for disabled login modules
+3. **Validate OAuth Bearer URLs** and prepare for future restrictions
+4. **Test Java 23 compatibility** if planning to use Java 23
+
+### Medium-term Planning (Before Kafka 4.0)
+
+1. **Migrate from ZooKeeper to KRaft** mode
+2. **Update deprecated client APIs** (pre-2.1 APIs)
+3. **Replace deprecated configurations:**
+ - `offsets.commit.required.acks`
+ - Log4J Appender
+ - `kafka.serializer.Decoder`
+4. **Plan Java version upgrade** (away from Java 11)
+
+### Configuration Updates Required
+
+```properties
+# New security-related system properties (3.9.1)
+-Dorg.apache.kafka.disallowed.login.modules=com.sun.security.auth.module.JndiLoginModule,com.sun.security.auth.module.LdapLoginModule
+-Dorg.apache.kafka.sasl.oauthbearer.allowed.urls=https://trusted-oauth-provider.com
+# Disable automatic config providers (3.7.0+)
+-Dorg.apache.kafka.automatic.config.providers=none
+# New configuration parameter (3.8.0)
+remote.fetch.max.wait.ms=5000
+```
+
+---
+
+## RISK ASSESSMENT
+
+### High Risk Changes
+
+- **SASL JAAS module restrictions** - May break existing authentication
+- **ZooKeeper deprecation** - Requires architectural changes
+- **Security vulnerabilities** - Must upgrade to 3.9.1
+
+### Medium Risk Changes
+
+- **Java version requirements** - Runtime environment changes
+- **Deprecated configurations** - Future breaking changes
+- **OAuth Bearer URL restrictions** - Future configuration requirements
+
+### Low Risk Changes
+
+- **New configuration parameters** - Optional additions
+- **New system properties** - Optional security enhancements
+
+---
+
+## TESTING RECOMMENDATIONS
+
+1. **Security Testing**
+ - Test SASL authentication with new module restrictions
+ - Validate OAuth Bearer URL configurations
+ - Verify ConfigProvider security fixes
+
+2. **Compatibility Testing**
+ - Test with Java 17+ and Java 23
+ - Verify deprecated API replacements
+ - Test new configuration parameters
+
+3. **Performance Testing**
+ - Benchmark with new `remote.fetch.max.wait.ms` setting
+ - Test KRaft mode performance vs ZooKeeper
+ - Validate security overhead impact
+
+---
+
+## CONCLUSION
+
+The upgrade from Kafka client 3.3.1 to 3.9.1 includes significant security improvements, architectural changes, and
+deprecations that will become breaking changes in Kafka 4.0. The most critical changes are:
+
+1. **Security fixes in 3.9.1** - Should be prioritized for immediate upgrade
+2. **ZooKeeper deprecation** - Requires planning for KRaft migration
+3. **Java version requirements** - Runtime environment updates needed
+4. **Configuration changes** - Various deprecated parameters need attention
+
+**Recommendation:** Plan a phased upgrade approach, starting with 3.9.1 for security fixes, followed by architectural
+changes (ZooKeeper → KRaft) and configuration updates before Kafka 4.0 release.
+
+---
+
+*This analysis is based on publicly available release notes, security advisories, and documentation. For the most
+current information, always refer to the official Apache Kafka release notes and upgrade guides.*
diff --git a/project.clj b/project.clj
index d3bd40c..86c7ff9 100644
--- a/project.clj
+++ b/project.clj
@@ -1,4 +1,4 @@
-(defproject com.appsflyer/ketu "2.0.0"
+(defproject com.appsflyer/ketu "2.1.0-SNAPSHOT"
:description "Clojure Apache Kafka client with core.async api"
:url "https://github.com/AppsFlyer/ketu"
:license {:name "Apache License, Version 2.0"
@@ -15,7 +15,7 @@
:dependencies [[org.clojure/clojure "1.11.1"]
[org.clojure/core.async "1.6.673"]
[expound "0.9.0"]
- [org.apache.kafka/kafka-clients "3.3.1"]
+ [org.apache.kafka/kafka-clients "3.9.1"]
[org.slf4j/slf4j-api "2.0.6"]]
:profiles {;; REPL, development and testing