why is my maven sub dependency version for spark connector package different from others

I am trying to use a pom file from a existing project and I am getting an error “Cannot resolve org.yaml:snakeyaml:1.15”

What I find out about this error is that the com.datastax.spark:spark-cassandra-connector_2.11:2.5.0 uses a couple dependencies and a couple levels down it is using snakeyaml:1.15 which is quarantined by company proxy. Is there a way to specify for a given maven dependency that I want to use snakeyaml:1.16?

One thing I do not understand is that I look into the reference project that is also using com.datastax.spark:spark-cassandra-connector_2.11:2.5.0, it is using the updated com.datastax.oss:java-driver-core-shaded:4.9.0, which no longer requires snakeyaml:1.15 where as mine uses the old com.datastax.oss:java-driver-core-shaded:4.5.0

Why is it working in that pom? we have the same maven listing version for com.datastax.spark:spark-cassandra-connector_2.11:2.5.0

I see it has some exclusions but none addresses the snake yaml version or any of its parent dependencies.

Is there another section of the pom file that addresses this I am missing? please advise.

My pom

<scala.compat.version>2.11</scala.compat.version>
<spark.cassandra.version>2.5.0</spark.cassandra.version>
<dependency>
  <groupId>com.datastax.spark</groupId>
  <artifactId>spark-cassandra-connector_${scala.compat.version}</artifactId>
  <version>${spark.cassandra.version}</version>
</dependency>

where it goes wrong

however another project is using the correct shaded version com.datastax.oss:java-driver-core-shaded:4.9.0, which eliminates the snake dependency

working pom

<scala.compat.version>2.11</scala.compat.version>
<spark.cassandra.version>2.5.0</spark.cassandra.version>
<dependency>
  <artifactId>spark-cassandra-connector_${scala.compat.version}</artifactId>
  <exclusions>
    <exclusion>
      <groupId>com.google.guava</groupId>
      <artifactId>guava</artifactId>
    </exclusion>
    <exclusion>
      <artifactId>netty-all</artifactId>
      <groupId>io.netty</groupId>
    </exclusion>
    <exclusion>
      <groupId>org.scala-lang</groupId>
      <artifactId>scala-library</artifactId>
    </exclusion>
  </exclusions>
  <groupId>com.datastax.spark</groupId>
  <version>${spark.cassandra.version}</version>
</dependency>

Answer

You add an entry your <dependencyManagement> section of your POM, where you specify the version of snakeyaml that you want.

This will override all transitive version definitions of snakeyaml.

Advertisement

Answer