Unable to send the logs to Splunk Enterprise local using log4j2

I’m using log4j2 and splunk within java to send logs into my Splunk Enterprise HEC (HTTP Event Collector) Splunk Enterprise is running in my local machine.

I’m doing all log4j2 configuration programmatically. (I know this is not the correct way to do this but I’m still doing this for learning purpose).

I tried to send the logs into Splunk Enterprise directly from postman with the same URL and token and it works fine, but when I tried to send the logs from java using log4j2 I don’t get anything in splunk.

My code is =>

import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.core.config.Configurator;
import org.apache.logging.log4j.core.config.builder.api.ConfigurationBuilder;
import org.apache.logging.log4j.core.config.builder.api.ConfigurationBuilderFactory;
import org.apache.logging.log4j.core.config.builder.impl.BuiltConfiguration;
import org.apache.logging.log4j.core.layout.PatternLayout;
import com.splunk.logging.*;

public class Main {
private static final Logger log;

static {
  configureLog4J();
  log = LogManager.getLogger(Main.class);
}
public static void configureLog4J() {
      ConfigurationBuilder<BuiltConfiguration> builder =
              ConfigurationBuilderFactory.newConfigurationBuilder();

      // configure a splunk appender
      builder.add(
          builder.newAppender("splunkH", "SplunkHttp")
              .add(
                  builder.newLayout(PatternLayout.class.getSimpleName())
                      .addAttribute(
                          "pattern",
                          "%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
                      )
              )
              .addAttribute("sourcetype", "log4j2")
              .addAttribute("index", "main")
              .addAttribute("url", "http://localhost:8088/services/collector") //I tried this url in postman and its working fine there
              .addAttribute("token", "xxx")
              .addAttribute("disableCertificateValidation", "true")
              
              
      );

      // configure the root logger
      builder.add(
          builder.newRootLogger(Level.INFO)
              .add(builder.newAppenderRef("splunkH"))
      );

      // apply the configuration
      Configurator.initialize(builder.build());

    }//end of configureLog4J

public static void main(String ar[]) {
    log.log(Level.INFO, "Hello from log4j2");
    
    log.log(Level.ERROR, "Error from log4j2");

}//end of main method
}//end of class

my POM file

<dependencies>
    <dependency>
        <groupId>com.splunk.logging</groupId>
        <artifactId>splunk-library-javalogging</artifactId>
        <version>1.11.4</version>
    </dependency>
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-core</artifactId>
        <version>2.11.2</version>
    </dependency>


    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-api</artifactId>
        <version>2.11.2</version>
    </dependency>
    <dependency>
        <groupId>com.splunk</groupId>
        <artifactId>splunk</artifactId>
        <version>1.6.5.0</version>
    </dependency>

</dependencies>

<repositories>
    <repository>
        <id>splunk-artifactory</id>
        <name>Splunk Releases</name>
        <url>https://splunk.jfrog.io/splunk/ext-releases-local</url>
    </repository>
</repositories>

I cannot see any logs in splunk. Did I miss something ?

Answer

Add .addAttribute("batch_size_count", "1") or make a loop producing 10 log messages, becasue that’s the default value of batch_size_count. This has been explained in splunk docs “Configure Log4j 2” section.

By the way, I reckon the services/collector endpoint should be used with JSON messages (e.g. .add(builder.newLayout("JSONLayout"))). Also, you are using a log4j2 version that has the Log4Shell (CVE-2021-44228) vulnerability. It has been fixed in 2.15.0, switch to anything between that and the newest version 2.17.2.

Finally, I share the sentiment of the answers to the question How to configure log4j 2.x purely programmatically? that log4j2 is troublesome to use when configured programmatically. I had issues with it in a cluster env and switching to file configuration solved all my problems.

Advertisement

Answer