Multiple dependency versions with Gradle 5

I need to support 2 different versions of the same library (to support a legacy version), es4hadoop for Apache Spark.

• Version 6.2.2(https://mvnrepository.com/artifact/org.elasticsearch/elasticsearch-spark-13_2.10/6.2.2)
• Version 6.3.2 (https://mvnrepository.com/artifact/org.elasticsearch/elasticsearch-spark-13_2.10/6.3.2)

Both versions have same dependencies (scala-lang and Spark).

Not sure at all about the naming of that, but I would like something like:

implementation(group: 'org.elasticsearch', name: 'elasticsearch-spark-13_' + scalaVersion, version:'6.2.2') {
     exclude group: "org.scala-lang"  
}

implementation(group: 'org.elasticsearch', name: 'elasticsearch-spark-13_' + scalaVersion, version:'6.3.2') {
     exclude group: "org.scala-lang"
     relocate org.elasticsearch org.elasticsearch6 // ???
}

so I can use both new and old elasticsearch library, in the same project / JVM.

I know already it’s possible to relocate lib with the shadowJar plugin, but is it possible to relocate a specific version?

Answer

Put one of the elasticsearch-spark dependencies into a subproject aa2 and relocate it. Then the other subproject aa1 can depend on aa2‘s shadow configuration.

// aa2/build.gradle
dependencies {
    implementation 'org.elasticsearch:elasticsearch-spark-13_2.10:6.2.2'
}

shadowJar {
    relocate "org.elasticsearch", "org.elasticsearch_v6_2_2"
}

// aa1/build.gradle
dependencies {
    implementation 'org.elasticsearch:elasticsearch-spark-13_2.10:6.3.2'
    implementation project(path: ':aa2', configuration: 'shadow')
}

You can now declare the same class in this way:

package com.github.chehsunliu.stackoverflow.q56332118;

import org.elasticsearch.spark.cfg.SparkSettings;

public class App {
  public static void main(String[] args) {
    System.out.println(SparkSettings.class);
    System.out.println(org.elasticsearch_v6_2_2.spark.cfg.SparkSettings.class);
  }
}

However, you should pay more attention to their transitive dependencies. You might also need to relocate them to make the direct dependencies work normally. Sometimes I will decompress the output JAR file and investigate these .class files to ensure the resolution correctness.

References

• Minimal project: https://github.com/chehsunliu/stackoverflow/tree/main/gradle/q56332118
• Decompiler https://java-decompiler.github.io/