Spring Boot Log4j2 configuration issue with log size maintenance

I have a spring boot application and using log4j2 to generate console and persists logs in a centos linux.

I wanted to maintain only 5mb of log files in archive.

But the problem is, my archived log files are 5mb in total. but my main console log which is saving in the main log file i.e wc-notification.out is going beyond 1mb.

so my disk gets full and it causes an issue.

The brute force method solution is: whenever restarting(hard stop and start) my spring boot application, the log is cleared in from wc-notification.out.

my log4j2 configuration xml file:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO" monitorInterval="30">
    <Properties>
        <Property name="LOG_PATTERN">
            [ %d{yyyy-MMM-dd HH:mm:ss a} ] - [%t] %-5level %logger{36} - %msg%n
        </Property>
    </Properties>
    <Appenders>
        <Console name="ConsoleAppender" target="SYSTEM_OUT" follow="true">
            <PatternLayout pattern="${LOG_PATTERN}"/>
        </Console>
        <RollingFile name="FileAppender" fileName="/home/ec2-user/apps/wc-notification-service/wc-notification.out"
                     filePattern="/home/ec2-user/apps/wc-notification-service/archives_test/wc-notification.out-%d{yyyy-MM-dd}-%i">
            <PatternLayout>
                <Pattern>${LOG_PATTERN}</Pattern>
            </PatternLayout>
            <Policies>
                <SizeBasedTriggeringPolicy size="1MB" />
            </Policies>
            <DefaultRolloverStrategy>
                <Delete basePath="logs" maxDepth="1">
                    <IfFileName glob="wc-notification.out-*.log" />
                    <IfLastModified age="1m" />
                </Delete>
            </DefaultRolloverStrategy>
        </RollingFile>
    </Appenders>
    <Loggers>
        <Root level="info">
            <!--<AppenderRef ref="ConsoleAppender" /> -->
            <AppenderRef ref="FileAppender" />
        </Root>
    </Loggers>
</Configuration>

somehow, the files are in range of 1mb, and the roll strategy is working, it is deleting the file

but, my disk space is still occupied with the space. what can be the reason?

Answer

As you are providing your own log4j2.xml configuration file, you are overwriting the Spring Boot default logging configuration, and it is safe to assume that it will be configuration used by Log4j2.

Your configuration is almost correct. If you want to achieve the desired behavior, I would suggest you the following changes:

• Be aware that you are pointing your Delete action basePath to the wrong location, it should point to the directory in which your logs are stored.
• The IfFileName glob pattern is wrong as well, it should match your logs filenames.
• Finally, your are using the IfLastModified condition. As its name implies, this condition has to do with the last modification date of the logs files, not with their size. Please, consider to use instead IfAccumulatedFileSize (or maybe IfAccumulatedFileCount). See the relevant documentation.

For these reasons, your logs are not being deleted correctly and the disk space occupied is greater than the desired amount. Without deletion, your log files are being rotated every 1 MB as configured by your SizeBasedTriggeringPolicy, and will keep until the value of the max attribute of DefaultRolloverStrategy, 7 by default, is reached, and always, plus the amount of your current log file.

In summary, please, try a configuration like this:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO" monitorInterval="30">
    <Properties>
        <Property name="LOG_PATTERN">
            [ %d{yyyy-MMM-dd HH:mm:ss a} ] - [%t] %-5level %logger{36} - %msg%n
        </Property>
    </Properties>
    <Appenders>
        <Console name="ConsoleAppender" target="SYSTEM_OUT" follow="true">
            <PatternLayout pattern="${LOG_PATTERN}"/>
        </Console>
        <RollingFile name="FileAppender" fileName="/home/ec2-user/apps/wc-notification-service/wc-notification.out"
                     filePattern="/home/ec2-user/apps/wc-notification-service/wc-notification.out-%d{yyyy-MM-dd}-%i">
            <PatternLayout>
                <Pattern>${LOG_PATTERN}</Pattern>
            </PatternLayout>
            <Policies>
                <SizeBasedTriggeringPolicy size="1MB" />
            </Policies>
            <DefaultRolloverStrategy>
                <Delete basePath="/home/ec2-user/apps/wc-notification-service">
                    <IfFileName glob="wc-notification.out-*" />
                    <IfAccumulatedFileSize exceeds="5 MB" />
                </Delete>
            </DefaultRolloverStrategy>
        </RollingFile>
    </Appenders>
    <Loggers>
        <Root level="info">
            <!--<AppenderRef ref="ConsoleAppender" /> -->
            <AppenderRef ref="FileAppender" />
        </Root>
    </Loggers>
</Configuration>

The solution relies in storing all your logs in the same location. It will affect your RollingFile filePattern attribute.

Please, be careful with the delete action, it can delete not only your log files, but all that matches your glob pattern.

In addition, although maybe irrelevant fo your use case, be aware that if the filePattern of your archive files ends with “.gz”, “.zip”, “.bz2”, among others, the resulting archive will be compressed using the compression scheme that matches the suffix, which can allow you to store more archives for the same space if required.

For your comments, it seems you encountered a problem when using large file sizes: please, see this bug, I think that clearly describes your problem.

My best advice will be to reduce the size of the logs files to one that it is properly working, or try a more recent version of the library.

I am aware that you are using Spring Boot to manage your dependencies: please, verify your maven tree and see the actual version library you are using and change it if necessary.