How to check if a particular keyword exists in list or file using spark scala
Solution:
//First we will check by list
val data = List("i am shah nawaz and my friend call me shah")
val sdata = sc.parallelize(data)
val flat_data = sdata.flatMap(x=>x.split(" "))
val words = flat_data.filter(x=>x.contains("shah"))
words.count()
//Second is by load data and find out specific keyword
val data = sc.textFile("file:////home/cloudera/words.txt")
val flat_data = data.flatMap(x=>x.split(" "))
val words = flat_data.filter(x=>x.contains("mango"))
words.count()
Comments
Post a Comment