reduce the internal log level for Log4j2 (with Kafka Appender)

I’m using Log4j2 (v2.17.2) to send information directly to kafka, and am using XML to create the configuration (as many articles mention that XML can handle a lot more configuration options – properties do not). The issue that I have is that my console is filled to the brim with irrelevant INFO log-lines (as an example):

[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version: 3.1.0
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId: 37edeed0777bacb3
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1652851625060
[pool-2-thread-1] INFO org.apache.kafka.clients.consumer.KafkaConsumer - [Consumer clientId=consumer-logInfo-1, groupId=logInfo] Subscribed to topic(s): logInfo
[pool-2-thread-1] INFO org.apache.kafka.clients.Metadata - [Consumer clientId=consumer-logInfo-1, groupId=logInfo] Cluster ID: 37Prit7oRwSnQ-CX5_Iwvw

I’ve tried all the techniques out from programmatically-change-log-level-in-log4j2 with no change to the logging:

Configurator.setLevel("org.apache.kafka", Level.WARN);

Has anyone had any luck with getting the log level to reduce from INFO? I really don’t want that level of information to trawl through to find the issues!

–edit–

I’ve gone through to explicitly set per class:

Configurator.setLevel("org.apache.kafka.clients.producer.ProducerConfig", Level.ERROR);
Configurator.setLevel("org.apache.kafka.clients.consumer.ConsumerConfig", Level.ERROR);

with the same results in the log:

[main] INFO org.apache.kafka.clients.producer.ProducerConfig - ProducerConfig values: 
[Thread-1] INFO org.apache.kafka.clients.consumer.ConsumerConfig - ConsumerConfig values:

Answer

It turns out that the answer is very simple: the kafka-clients-3.1.0.jar (required to get the kafka appender to work) uses slf4j for its logging, so doesn’t respond to any changes to log4j levels!

The following line fixes the issue:

System.setProperty(org.slf4j.impl.SimpleLogger.DEFAULT_LOG_LEVEL_KEY, "ERROR");

Advertisement

Answer