Advertisement

Tag: rdd

Using createOrReplaceTempView to replace a temp view not working as expected

I have a dataset something similar to this My spark code is I am trying to replace the people view by calling createOrReplaceTempView but i get the following error as below How do I replace the view in spark? Answer So I got the solution to the above question by the following code

Spark read file from S3 using sc.textFile (“s3n://…)

Trying to read a file located in S3 using spark-shell: scala> val myRdd = sc.textFile(“s3n://myBucket/myFile1.log”) lyrics: org.apache.spark.rdd.RDD[String] = s3n://myBucket/myFile1.log MappedRDD[…

Advertisement