Alter client quotas using kafka-python - apache-kafka

Question

I'm using kafka-python library to manage my kafka cluster. I want to add the quotas features (alter/describe), these features are not yet implented in kafka-python library but they are available in kafka protocol doc here.
AlterClientQuotas API (Key: 49):
Requests:
AlterClientQuotas Request (Version: 0) => [entries] validate_only
entries => [entity] [ops]
entity => entity_type entity_name
entity_type => STRING
entity_name => NULLABLE_STRING
ops => key value remove
key => STRING
value => FLOAT64
remove => BOOLEAN
validate_only => BOOLEAN
Responses:
AlterClientQuotas Response (Version: 0) => throttle_time_ms [entries]
throttle_time_ms => INT32
entries => error_code error_message [entity]
error_code => INT16
error_message => NULLABLE_STRING
entity => entity_type entity_name
entity_type => STRING
entity_name => NULLABLE_STRING
To implement these features I'm adding the following code
kafka/protocol/admin.py
from kafka.protocol.api import Request, Response
class AlterClientQuotasResponse_v0(Response):
API_KEY = 49
API_VERSION = 0
SCHEMA = Schema(
("throttle_time_ms", Int32),
(
"entries",
Array(
("error_code", Int16),
("error_message", String),
("match", CompactString("utf-8")),
(
"entity",
Array(
("entity_type", String("utf-8")),
("entity_name", String("utf-8")),
),
),
),
),
)
class AlterClientQuotasResponse_v1(Response):
API_KEY = 49
API_VERSION = 0
SCHEMA = Schema(
("throttle_time_ms", Int32),
(
"entries",
Array(
("error_code", Int16),
("error_message", String),
("match", CompactString("utf-8")),
(
"entity",
Array(
("entity_type", String("utf-8")),
("entity_name", String("utf-8")),
),
),
("tags", TaggedFields),
),
),
)
class AlterClientQuotasRequest_v0(Request):
API_KEY = 49
API_VERSION = 0
RESPONSE_TYPE = AlterClientQuotasResponse_v0
SCHEMA = Schema(
(
"entries", Array(
("entity_type", String("utf-8")),
("entity_name", String("utf-8")),
)
),
(
"ops", Array(
("key", String("utf-8")),
("value", Float64),
("remove", Boolean),
)
),
("validate_only", Boolean)
)
class AlterClientQuotasRequest_v1(Request):
API_KEY = 49
API_VERSION = 0
RESPONSE_TYPE = AlterClientQuotasResponse_v1
SCHEMA = Schema(
(
"entries", Array(
("entity_type", String("utf-8")),
("entity_name", String("utf-8")),
)
),
(
"ops", Array(
("key", String("utf-8")),
("value", Float64),
("remove", Boolean),
)
),
("validate_only", Boolean),
("tags", TaggedFields),
)
AlterClientQuotasResponse = [
AlterClientQuotasResponse_v0,
AlterClientQuotasResponse_v1
]
AlterClientQuotasRequest = [
AlterClientQuotasRequest_v0,
AlterClientQuotasRequest_v1
]
kafka/admin/client.py
request = AlterClientQuotasRequest[0](
entries=[("user", "<...>.producer"),],
ops=[("producer_byte_rate", 1024, False),],
validate_only=True,
#tags={}
)
print("--> Quota Request :", request)
future = self._send_request_to_node(self._client.least_loaded_node(), request)
self._wait_for_futures([future])
print("-->>> FUTURE Value", future.value)
Running the code below gets the follwing errors :
client output
File /<...>/py310/lib/python3.10/site-packages/kafka/admin/client.py:1342, in KafkaAdminClient._wait_for_futures(self, futures)
1339 self._client.poll(future=future)
1341 if future.failed():
-> 1342 raise future.exception
KafkaConnectionError: KafkaConnectionError: socket disconnected
server output
Feb 03 11:24:25 broker-05: org.apache.kafka.common.errors.InvalidRequestException: Error getting request for apiKey: ALTER_CLIENT_QUOTAS, apiVersion: 0, connectionId: , listenerName: ListenerName(SECURED), principal: User:
Feb 03 11:24:25 broker-05 Caused by: java.lang.RuntimeException: Tried to allocate a collection of size 292211, but there are only 71 bytes remaining.
Feb 03 11:24:25 broker-05 at org.apache.kafka.common.message.AlterClientQuotasRequestData$EntryData.read(AlterClientQuotasRequestData.java:347)
Feb 03 11:24:25 broker-05 at org.apache.kafka.common.message.AlterClientQuotasRequestData$EntryData.(AlterClientQuotasRequestData.java:300)
Feb 03 11:24:25 broker-05 at org.apache.kafka.common.message.AlterClientQuotasRequestData.read(AlterClientQuotasRequestData.java:125)
Feb 03 11:24:25 broker-05 at org.apache.kafka.common.message.AlterClientQuotasRequestData.(AlterClientQuotasRequestData.java:73)
Feb 03 11:24:25 broker-05 at org.apache.kafka.common.requests.AlterClientQuotasRequest.parse(AlterClientQuotasRequest.java:142)
Feb 03 11:24:25 broker-05 at org.apache.kafka.common.requests.AbstractRequest.doParseRequest(AbstractRequest.java:269)
Feb 03 11:24:25 broker-05 at org.apache.kafka.common.requests.AbstractRequest.parseRequest(AbstractRequest.java:165)
Feb 03 11:24:25 broker-05 at org.apache.kafka.common.requests.RequestContext.parseRequest(RequestContext.java:95)
Feb 03 11:24:25 broker-05 at kafka.network.RequestChannel$Request.(RequestChannel.scala:101)
Feb 03 11:24:25 broker-05 at kafka.network.Processor.$anonfun$processCompletedReceives$1(SocketServer.scala:1096)
Feb 03 11:24:25 broker-05 at java.base/java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608)
Feb 03 11:24:25 broker-05 at kafka.network.Processor.processCompletedReceives(SocketServer.scala:1074)
Feb 03 11:24:25 broker-05 at kafka.network.Processor.run(SocketServer.scala:960)
Feb 03 11:24:25 broker-05 at java.base/java.lang.Thread.run(Thread.java:829)
Has anyone been able to implement this ?
help please ?
Thank you !