Kafka3.7 on CentOS Stream 9 (X86-64) with support by Hfami

AWS-Marketplace

https://aws.amazon.com/marketplace/pp/prodview-podpwgven2als

Simple usage example

1. Confirm Kafka and Zookeeper Service Status

First, ensure that the Kafka and Zookeeper services are running:
- sudo systemctl status zookeeper
- sudo systemctl status kafka

If the services are not running, you can start them with the following commands:
- sudo systemctl start zookeeper
- sudo systemctl start kafka

2. Create a Kafka Topic

Kafka uses topics to organize messages. You can create a new topic using Kafka's built-in command-line tool.
- /opt/kafka/bin/kafka-topics.sh --create --topic my-topic --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1

3. List Kafka Topics

Verify if the topic was created successfully:
- /opt/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092

4. Produce Messages

Use the Kafka console producer to send messages to the topic my-topic:
- /opt/kafka/bin/kafka-console-producer.sh --topic my-topic --bootstrap-server localhost:9092

Enter a message and press Enter, for example:

Hello world!

5. Consume Messages

Use the Kafka console consumer to read messages from my-topic:
- /opt/kafka/bin/kafka-console-consumer.sh --topic my-topic --from-beginning --bootstrap-server localhost:9092

You should see the previously sent message, for example:

Hello world!

6. Manage Kafka

You can use other Kafka command-line tools to manage Kafka, such as viewing topic details, deleting topics, etc.

View Topic Details:
- /opt/kafka/bin/kafka-topics.sh --describe --topic my-topic --bootstrap-server localhost:9092

Delete a Topic:
- /opt/kafka/bin/kafka-topics.sh --delete --topic my-topic --bootstrap-server localhost:9092

7. Use Kafka Consumer Groups

Kafka supports consumer groups, allowing multiple consumers in a group to read from different partitions of the same topic.

Start Multiple Consumers:

You can start multiple consumers in different terminal windows specifying the same consumer group:
- /opt/kafka/bin/kafka-console-consumer.sh --topic my-topic --group my-group --bootstrap-server localhost:9092

As you send more messages, they will be distributed among these consumers.

8. Configure and Monitor Kafka

You can configure Kafka's server.properties file to adjust Kafka's performance and behavior. Common configuration options include:

• broker.id: Unique identifier for each Kafka broker.
  
• log.dirs: Directory where Kafka log data is stored.
  
• zookeeper.connect: Address to connect to the Zookeeper ensemble.
  

9.Related commands