JUnit 5 @EnabledIfSystemProperty doesn’t work as expected

I migrated my test from JUnit 4 to JUnit 5. All works fine but the translation of my previous annotation:

@IfProfileValue(name = "run.import.tests", values = {"true"})

into

@EnabledIfSystemProperty(named = "run.import.tests", matches = "true")

doesn’t work as expected. Before the migration I runned my tests passing the argument

-Drun.import.tests=true

only if I passed it they were runned. With Junit 5, even with annotation @EnabledIfSystemProperty(named = "run.import.tests", matches = "true") the test is runned even if the argument run.import.tests is not set.

Am I doing something wrong?

Answer

To make it work an “opposite” annotation has to be added, so both of them together look like this:

@EnabledIfSystemProperty(named = "run.import.tests", matches = "true")
@DisabledIfSystemProperty(named = "run.import.tests", matches = "(?!true)")

I’ve checked it and the test class is disabled if the run.import.tests property is not set or if it is set to any other value than true; if the value is set to true – the test class is not disabled.

---

Curiously, the documentation of @EnabledIfSystemProperty states:

If the specified system property is undefined, the annotated class or method will be disabled.

Yet it does not work that way and it may be a bug. I will try and debug the JUnit classes and if I create an issue on their GitHub, I will link it here.

---

I’ve gone through the code and tested it a few more times – here is the summary:

1. When I run the test using Maven (mvn test), the annotation @EnabledIfSystemProperty alone works fine – the test is run only when I add -Drun.import.tests=true argument. @DisabledIfSystemProperty is not needed in that case.
2. When I run the test using IntelliJ’s Run XxxTest handling of the property worked fine only if both annotations were present. After some debugging I came across JupiterTestEngine – a class that is run by external launchers (Maven, IntelliJ, any other). It seems that IntelliJ adds a property to it’s test launcher: junit.jupiter.conditions.deactivate, which is usually useful – thanks to that we can run even tests that are disabled with conditional annotations locally, ignoring them. The value of the property is org.junit.*Enabled*Condition when @DisabledIfSystemProperty is not present and org.junit.*Disabled*Condition when it is – the conditions are JUnit’s extensions that resolve disabled state for the test.

The functionality described in (2) is usually useful, but in your case it made it look like the annotation is not working. It actually is working, but IntelliJ just bypasses it.