原创, 服务器,

使用Kafka-acls授权时报错InvalidACL for kafka-acl解决方法

给Kafka Topic 授权读写操作，使用 kafka-acls

之前，我在CentOS上部署Kafka服务，使用各种命令都是正常的。

./kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181  --list

但是最近，我开始使用Docker部署Kafka服务，结果就出现了各种问题。docker-compose.yml代码如下

version: '3.1'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:5.1.1
    hostname: zookeeper
    container_name: zookeeper
    restart: always
    ports:
      - 2181:2181
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_MAXCLIENTCNXNS: 0
      ZOOKEEPER_AUTHPROVIDER.1: org.apache.zookeeper.server.auth.SASLAuthenticationProvider
      ZOOKEEPER_REQUIRECLIENTAUTHSCHEME: sasl
      ZOOKEEPER_JAASLOGINRENEW: 3600000
      TZ: Asia/Shanghai
      KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/kafka_zoo_jaas.conf -Dzookeeper.sasl.serverconfig=ZKServer
    volumes:
      - ./config:/etc/kafka
      - ./bin/zookeeper-server-start:/usr/bin/zookeeper-server-start
      - /data/kafka-data/zookeeper:/var/lib/zookeeper/data
      - /data/logs/kafka-zookeeper:/var/lib/zookeeper/log
  kafka:
    image: confluentinc/cp-kafka:5.1.1
    hostname: broker
    container_name: kafka
    restart: always
    depends_on:
      - zookeeper
    ports:
      - 9092:9092
    environment:
      TZ: Asia/Shanghai
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181/kafka'
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_LISTENERS: SASL_PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_LISTENERS: SASL_PLAINTEXT://1.1.1.1:9092
      KAFKA_SECURITY_INTER_BROKER_PROTOCOL: SASL_PLAINTEXT
      KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
      KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
      KAFKA_AUTHORIZER_CLASS_NAME: kafka.security.auth.SimpleAclAuthorizer
      KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf
      KAFKA_SUPER_USERS: User:admin;User:producer
    volumes:
      - ./config:/etc/kafka
      - ./bin/kafka-console-producer:/usr/bin/kafka-console-producer
      - ./bin/kafka-console-consumer:/usr/bin/kafka-console-consumer
      - ./bin/kafka-server-start:/usr/bin/kafka-server-start
      - /data/kafka-data/kafka:/var/lib/kafka
  kafka-manager:
    image: kafkamanager/kafka-manager
    hostname: kafka-manager
    container_name: kafka-manager
    restart: always
    depends_on:
      - zookeeper
      - kafka
    ports:
      - 9000:9000
    environment:
      TZ: Asia/Shanghai
      ZK_HOSTS: zookeeper:2181
      KAFKA_MANAGER_AUTH_ENABLED: 'true'
      KAFKA_MANAGER_USERNAME: admin
      KAFKA_MANAGER_PASSWORD: secretpassword

我进入到Kafka容器中，想要跑授权命令。

# 进入kafka容器
$ docker exec -it kafka bash
$ cd /usr/bin/

# 查看授权列表
$ ./kafka-acls --authorizer-properties zookeeper.connect=zookeeper:2181/kafka --list

结果却看到了如下错误信息。

Error while executing ACL command: KeeperErrorCode = InvalidACL for /kafka-acl
org.apache.zookeeper.KeeperException$InvalidACLException: KeeperErrorCode = InvalidACL for /kafka-acl
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:124)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
        at kafka.zookeeper.AsyncResponse.maybeThrow(ZooKeeperClient.scala:546)
        at kafka.zk.KafkaZkClient.createRecursive(KafkaZkClient.scala:1559)
        at kafka.zk.KafkaZkClient$$anonfun$createAclPaths$1.apply(KafkaZkClient.scala:1059)
        at kafka.zk.KafkaZkClient$$anonfun$createAclPaths$1.apply(KafkaZkClient.scala:1058)
        at scala.collection.Iterator$class.foreach(Iterator.scala:891)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
        at scala.collection.MapLike$DefaultValuesIterable.foreach(MapLike.scala:206)
        at kafka.zk.KafkaZkClient.createAclPaths(KafkaZkClient.scala:1058)
        at kafka.security.auth.SimpleAclAuthorizer.configure(SimpleAclAuthorizer.scala:102)
        at kafka.admin.AclCommand$AuthorizerService.withAuthorizer(AclCommand.scala:213)
        at kafka.admin.AclCommand$AuthorizerService.listAcls(AclCommand.scala:250)
        at kafka.admin.AclCommand$.main(AclCommand.scala:74)
        at kafka.admin.AclCommand.main(AclCommand.scala)

我开始以为是Zookeeper的问题，比较原来的服务器上zookeeper 记录和docker中的zookeeper 没有发现任何异常， 于是，我开始各种查询资料和参考命令行参数，终于发现问题所在——请求时没有带授权信息。 那么，要怎么样带上授权信息呢？这里有一个重要的参数：--command-config。官网上使用一个文件名为adminclient-configs.conf的配置文件，可是我在GitHub上找了半天也没找到啊！

最后，好心的chatgpt帮我找到了这个神秘的配置文件，并给出了配置内容。感谢chatgpt！配置如下

# /etc/kafka/adminclient-configs.conf

bootstrap.servers=localhost:9092

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="secretpassword";

所以 正确的使用 kafka-acls 的命令如下

# 查看授权列表
$ /usr/bin/kafka-acls --bootstrap-server localhost:9092 --command-config /etc/kafka/adminclient-configs.conf --list

# 查看某个topic授权
$ /usr/bin/kafka-acls --bootstrap-server localhost:9092 --command-config /etc/kafka/adminclient-configs.conf --list --topic topicName

# 给topic设置某个IP的读权限
$ /usr/bin/kafka-acls --bootstrap-server localhost:9092 --command-config /etc/kafka/adminclient-configs.conf --add --allow-principal User:'*' --allow-host 39.39.39.39  --operation Read --topic topicName

现在，我终于可以正常授权了！如果你也遇到了类似的问题，可以试试我的方法哦。

总之，学习过程中遇到问题很正常，重要的是我们要勇于面对，不断探索，最终解决问题。加油！（chatgpt帮忙整理出的文章，大家凑活看吧）

(731)

Related Post