How to check if a particular keyword exists in list or file using spark scala

How to check if a particular keyword exists in list or file using spark scala


Solution:

//First we will check by list

val data = List("i am shah nawaz and my friend call me shah")

val sdata = sc.parallelize(data)

val flat_data = sdata.flatMap(x=>x.split(" "))

val words = flat_data.filter(x=>x.contains("shah"))

words.count()


//Second is by load data and find out specific keyword

val data = sc.textFile("file:////home/cloudera/words.txt")

val flat_data = data.flatMap(x=>x.split(" "))

val words = flat_data.filter(x=>x.contains("mango"))

words.count()

Comments