Kafka Producer/Consumer Properties to JSON

Kafka Configuration Properties

Apache Kafka clients (producers, consumers, admin clients) are configured via .properties files or Java Properties objects. These configurations control serialization, batching, acknowledgment behavior, and security settings.

Example: Kafka Producer Properties

# Bootstrap servers
spring.kafka.bootstrap-servers=kafka-1:9092,kafka-2:9092,kafka-3:9092

# Producer Configuration
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.acks=all
spring.kafka.producer.retries=3
spring.kafka.producer.batch-size=16384
spring.kafka.producer.linger-ms=5
spring.kafka.producer.buffer-memory=33554432
spring.kafka.producer.compression-type=snappy

# Consumer Configuration
spring.kafka.consumer.group-id=my-consumer-group
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.consumer.enable-auto-commit=false
spring.kafka.consumer.max-poll-records=500
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer

# SSL Configuration
spring.kafka.ssl.trust-store-location=/etc/kafka/truststore.jks
spring.kafka.ssl.trust-store-password=changeit
spring.kafka.ssl.key-store-location=/etc/kafka/keystore.jks
spring.kafka.ssl.key-store-password=changeit
spring.kafka.properties.security.protocol=SSL

Nested JSON Structure

The JSON output groups producer, consumer, and SSL settings into logical subtrees, making it easy to review each component independently and compare settings across different Kafka clusters or environments.

Why Convert to JSON?

• Configuration management: Store Kafka config in JSON-based systems (Consul, Vault)
• Docker/Kubernetes: Generate environment-specific config maps from properties
• Documentation: Create readable configuration reference docs
• Validation: Use JSON Schema to validate Kafka configuration