I am running Hadoop 2.8.1 and Hive 2.3.0 I am tring to read values from an a table created in Hive and the current exception is
java.lang.ClassNotFoundException: org.apache.hadoop.tracing.SpanReceiverHost at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
And here is the code that I have used to read the tables
public static final String HIVEURL = "jdbc:hive2://localhost:10000"; public static final String DB_NAME = "default"; public static final String TABLE_NAME = "order_line"; public static void main(String[] args) throws Exception { HiveConf hiveConf = new HiveConf(); //hiveConf.setVar(HiveConf.ConfVars.METASTOREURIS, HIVEURL); HiveMetaStoreClient hiveClient = new HiveMetaStoreClient(hiveConf); Job job =Job.getInstance(); TaskAttemptContext ctx = new TaskAttemptContextImpl(job.getConfiguration(), new TaskAttemptID()); HCatInputFormat hcif = HCatInputFormat.setInput(job, DB_NAME, TABLE_NAME); HCatSchema allCols = hcif.getTableSchema(job.getConfiguration()); List<HCatFieldSchema> usedList = new ArrayList<>(); usedList.add(allCols.get(2)); // por ex... HCatSchema someCols = new HCatSchema(usedList); hcif.setOutputSchema(job, someCols); for(InputSplit split: hcif.getSplits(job)) { RecordReader<WritableComparable, HCatRecord> rr = hcif.createRecordReader(split,ctx); rr.initialize(split, ctx); while(rr.nextKeyValue()) { HCatRecord record = rr.getCurrentValue(); // usar record.get(...) para obter a coluna... //Object o = record.get(1); //System.out.println(o.toString()); } rr.close(); } hiveClient.close(); }
And here it is the Pom file that I have used
org.apache.hive.hcatalog hive-hcatalog-core 2.3.0 org.apache.hive.hcatalog hive-hcatalog 0.13.1-cdh5.3.5 org.apache.hive hive-common 2.3.0
<dependency> <groupId>org.apache.hive</groupId> <artifactId>hive</artifactId> <version>0.13.1-cdh5.3.5</version> </dependency> <dependency> <groupId>org.apache.hive</groupId> <artifactId>hive-metastore</artifactId> <version>2.3.0</version> </dependency> <!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-common --> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>2.8.1</version> </dependency> <!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-core --> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-core</artifactId> <version>2.6.0-mr1-cdh5.12.1</version> <type>pom</type> </dependency> <dependency> <groupId>org.apache.thrift</groupId> <artifactId>libthrift</artifactId> <version>0.9.3</version> </dependency> </dependencies>
Advertisement
Answer
I can’t see what actually causes the loadClass
from the stack trace snippet, but it appears that the class doesn’t actually exist in the version, 2.8.1, of hadoop-common
you’re using. It seems to have vanished somewhere after 2.7.2
It, or something with the same name, is in the hbase source
Have you got a mix’n’match of versions going on?