LaVOZs

The World’s Largest Online Community for Developers

'; scala - I want to filter the Lines read form text file with set of the key words - LavOzs.Com

I have written below code, it is working for the one word but when I give the seq variable term I am not getting the output, can anyone tell me how to solve this.

val term = List("Achieving","Making")

val sc = new SparkContext("local[*]","Filter_lines")

val Lines = sc.textFile("../book.txt")

val filter_Lines = Lines.filter(l => l.contains("Making")).collect()

filter_Lines.foreach(println)

Try this -

Lines.filter(l => term.exists(t => l.contains(t))).foreach(println)

exists function on the collection accepts a function that returns true if the "l" contains any of the "t" terms.

Related
scala not running on Centos Linux box
How to read multiple text files into a single RDD?
How to read a file and store words to a list in scala?
Splitting strings in Apache Spark using Scala
Scala Seq for Spark in Java?
How to load extra spark properties using --properties-file option in spark yarn cluster mode?
Parse and Show the data of a JSON file in Scala | Meaning of .config(“spark.some.config.option”, “some-value”).getOrCreate()
List frequently occurring words across different rows