Can you plese share the command to allow READ for a userid for all consumer group?
Below works fine. But I want to grant access for all consumergroup for that username
./confluent iam acl create --allow --principal User:userName --operation READ --consumer-group groupName --kafka-cluster-id UjOnbv9hAsdAk7Tuk4RX4w
There are 2 options to work with ACLs for multiple resources:
Use asterisk '*' instead of group or topic names. In Confluent docs examples it is used with --topics flag, it should work with groups as well.
confluent iam acl create --allow --principal User:User1 --operation WRITE
--consumer-group '*' --kafka-cluster-id <kafka-cluster-id>
To select subset of groups or topics, use --prefix flag.
confluent iam acl create --allow --principal User:User1 --operation WRITE
--consumer-group 'prefixString' --prefix --kafka-cluster-id <kafka-cluster-id>
Review all the ACLs for the specified cluster that include allow permissions for the user:
confluent iam acl list --kafka-cluster-id <kafka-cluster-id> --allow --principal User:Jane
Example of how prefixed and asterisk are displayed in the liast of ACLs:
Related
How can authorize any consumer group to access a topic from a user that has permission in the ACL?
I am publishing data into topic test-1. And I authorized user-1 to have READ access to the Kafka ACL. But when I try to consume from the topic, I am getting a GROUP AUTHORIZATION EXCEPTION.
Is there a way to authorize any group on a topic for a particular user?
Yes you can authorize using wildcards so something like:
kafka-acls --bootstrap-server $host:$port --command-config adminclient-configs.conf --add --allow-principal \
User:$your_user --operation All --topic '$topic_name' --group '*'
You could use wildcard on topic also but not recommended and you can also adjust the operations more specifically(recommended), i.e - READ instead of using 'All' as in the example above.
I'm not able to find Option to control the Topic level permission in Cloud Based Free Trial of Confluent Platform. Can you please suggest on how to configure ?
Confluent Cloud role-based access control (RBAC) provides a method to control access to an organization, environment, or cluster configuration using predefined roles. RBAC enables enterprises to protect their production environment by isolating user and service account access and allowing for the delegation of authorization to the appropriate business units and teams.
To control access to specific resources within a cluster, such as Kafka topics or ksqlDB applications, continue to use ACLs.
RBAC does not provide granular support for Kafka resources (like topics), nor does it provide granular access control for individual connectors, ksqlDB applications, and schema subjects.
ACL are not manged by the Control Center
https://docs.confluent.io/cloud/current/access-management/cloud-rbac.htm
ACLs are managed using the Confluent Cloud CLI. For a complete list of Kafka ACLs, see Authorization using ACLs.
https://docs.confluent.io/ccloud-cli/current/command-reference/kafka/acl/index.html#ccloud-kafka-acl
ccloud kafka acl create --allow --service-account sa-55555 --operation READ --operation DESCRIBE --consumer-group java_example_group_1
ccloud kafka acl create --allow --service-account sa-55555 --operation READ --operation DESCRIBE --topic '*'
I am trying to set up Kafka, where each user have several topics, but each topic may be consumed with any number of consumer group by the user the topic belongs to.
Kafka server version used: kafka_2.12-2.4.0 (Commit:77a89fcf8d7fa018)
Kafka client version used: confluent kafka 1.2.2
In Kafka ACL have successfully configured users, so they can only access only their own topic. I'm struggling to set up group permissions in such a way where each user can use any number of consumer groups just for their own topic without seeing what consumer groups others have.
The following enables every user to use any consumer group:
bin/kafka-acls.sh localhost:9092 --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=zookeeper.address --add --allow-principal User:* --operation Read --group '*'
However, according to https://docs.confluent.io/current/kafka/authorization.html Read operation implicitly grants Describe operation. As Describe operation includes access to 'ListGroup' API, which I do not want my users to be able to do, I executed the following:
bin/kafka-acls.sh localhost:9092 --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=zookeeper.address --add --deny-principal User:* --operation Describe --group '*'
The two commands above result in the following ACLs:
Current ACLs for resource `Group:LITERAL:*`:
User:* has Deny permission for operations: Describe from hosts: *
User:* has Allow permission for operations: Read from hosts: *
The problem with this is I'm getting the following exception:
Confluent.Kafka.ConsumeException: Broker: Group authorization failed
Which leads me to believe I'm either trying to achieve the impossible or trying it wrong.
TLDR: Is it possible to set up Kafka ACLs to allow using any consumer group without also granting ListGroups API permission at the same time?
Thanks for any answer.
For now decided to use prefix. Works well enough.
For those wondering how to do this:
bin/kafka-acls.sh localhost:9092 --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=zookeeper.address --add --allow-principal User:XYZ --operation Read --group 'ABC-' --resource-pattern-type prefixed
This piece of code will allow user 'XYZ' to use any consumer group starting with 'ABC-', like 'ABC-123'
I am collecting the data from different resources, each resource has one specific topic for each client.
I want to give the access for each user only to the corresponding topic, so they can't have access to all the topics.
I am working with Kafka 0.10 and I am using Kafka tools.
there is solution?
You need to configure Authorisation using ACL.
How to enable ACL:
In your server.properties file, you need to create an Authorizer by adding the following line:
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
Now you need to follow the docs in order to properly configure ACL based on your use cases.
Adding ACLs
Now once everything is in place, let's assume you have a topic called testTopic to which you want to grant read and write access only to user called Bob from a host with IP 197.5.6.1:
bin/kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 \
--add \
--allow-principal User:'Bob' --allow-host '197.5.6.1' \
--operation Read --operation Write \
--topic testTopic \
I've a 3-node unsecured kafka(v0.10.2.1) cluster with topic auto creation and deletion disabled with the following in server.properties
auto.create.topics.enable=false
delete.topic.enable=true
Topics are then created/altered on the cluster using bin/kafka-topics.sh. However, it looks like anyone can create topics on the cluster once they know the end points.
Is there a way to lock down topic creation/alteration to specific hosts to prevent abuses?
Edit 1:
Since ACL was suggested, I tried to restrict topic creation to select hosts using kafka-acls.sh.
I restarted the brokers after adding the following to server.properties, .
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
allow.everyone.if.no.acl.found=true
I tried the below to restrict topic creation on localhost.
bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:* --cluster --operation Create --allow-host 127.0.0.1
However, I was still able to create topics from an other host using kafka-topics.sh with the right endpoints. Is it the case that ACLs can't be used without authentication?
You need to use access control lists (ACLs) to restrict such operations and that implies knowing who the caller is, so you need kafka to be secured by an authentication mechanism in the first place.
ACLs: http://kafka.apache.org/documentation.html#security_authz
Authentication can be done using SSL or SASL or by plugging in a custom provider, see the preceding sections of the same document.
Disabling auto-creation is not an access control mechanism, it only means that trying to produce to or consume from a topic will not create it automatically.