How to use Kafka with SSL via logback appander?

I use this logback appender to send logs to Kafka:

https://github.com/danielwegener/logback-kafka-appender

When Kafka was PLAINTEXT everything worked correctly. But when Kafka changed to SSL, it is not possible to send messages. I did not find the necessary information in readme.md. Has anyone had this setup experience? Or maybe use something else?

<topic>TEST_TOPIC_FOR_OS</topic>
<keyingStrategy class="com.github.danielwegener.logback.kafka.keying.NoKeyKeyingStrategy"/>
<deliveryStrategy class="com.github.danielwegener.logback.kafka.delivery.AsynchronousDeliveryStrategy">
</deliveryStrategy>
<producerConfig>metadata.fetch.timeout.ms=99999999999</producerConfig>
<producerConfig>bootstrap.servers=KAFKA BROKER HOST</producerConfig>
<producerConfig>acks=0</producerConfig>
<producerConfig>linger.ms=1000</producerConfig>
<producerConfig>buffer.memory=16777216</producerConfig>
<producerConfig>max.block.ms=100</producerConfig>
<producerConfig>retries=2</producerConfig>
<producerConfig>client.id=${HOSTNAME}-${CONTEXT_NAME}-logback</producerConfig>
<producerConfig>compression.type=none</producerConfig>
<producerConfig>security.protocol=SSL</producerConfig>
<producerConfig>ssl.keystore.location= path_to_jks</producerConfig>
<producerConfig>ssl.keystore.password=PASSWORD</producerConfig>
<producerConfig>ssl.truststore.location=path_to_jks </producerConfig>
<producerConfig>ssl.truststore.password=PASSWORD </producerConfig>
<producerConfig>ssl.endpoint.identification.algorithm=</producerConfig>
<producerConfig>ssl.protocol=TLSv1.1</producerConfig>

JavaScript
​x
 
<topic>TEST_TOPIC_FOR_OS</topic><keyingStrategy class="com.github.danielwegener.logback.kafka.keying.NoKeyKeyingStrategy"/><deliveryStrategy class="com.github.danielwegener.logback.kafka.delivery.AsynchronousDeliveryStrategy"></deliveryStrategy><producerConfig>metadata.fetch.timeout.ms=99999999999</producerConfig><producerConfig>bootstrap.servers=KAFKA BROKER HOST</producerConfig><producerConfig>acks=0</producerConfig><producerConfig>linger.ms=1000</producerConfig><producerConfig>buffer.memory=16777216</producerConfig><producerConfig>max.block.ms=100</producerConfig><producerConfig>retries=2</producerConfig><producerConfig>client.id=${HOSTNAME}-${CONTEXT_NAME}-logback</producerConfig><producerConfig>compression.type=none</producerConfig><producerConfig>security.protocol=SSL</producerConfig><producerConfig>ssl.keystore.location= path_to_jks</producerConfig><producerConfig>ssl.keystore.password=PASSWORD</producerConfig><producerConfig>ssl.truststore.location=path_to_jks </producerConfig><producerConfig>ssl.truststore.password=PASSWORD </producerConfig><producerConfig>ssl.endpoint.identification.algorithm=</producerConfig><producerConfig>ssl.protocol=TLSv1.1</producerConfig>​

For any existing topic, I get an error:

12:05:49.505 [kafka-producer-network-thread | host-default-logback] route: DEBUG o.a.k.clients.producer.KafkaProducer breadcrumbId: - [Producer clientId=host-default-logback] Exception occurred during message send:
org.apache.kafka.common.errors.TimeoutException: Topic TEST_TOPIC_FOR_OS not present in metadata after 100 ms.

JavaScript
 
12:05:49.505 [kafka-producer-network-thread | host-default-logback] route: DEBUG o.a.k.clients.producer.KafkaProducer breadcrumbId: - [Producer clientId=host-default-logback] Exception occurred during message send:org.apache.kafka.common.errors.TimeoutException: Topic TEST_TOPIC_FOR_OS not present in metadata after 100 ms.​

The application itself works correctly with this kafka and topic

Answer

The problem went away with the upgrade of appender to 0.2.0

Advertisement

Answer