How to replace some data from all column in dataframe in spark scala

How to replace some data from all column in dataframe in spark scala

Input :

Output:



Solution:

#First we need to read data, well

val df = spark.read.format("csv").option("header","True").load("file:///D:/idname.txt")

df.show()


#we just make function 

def repdata(col:Column):Column=when(col==="Shah",null).otherwise(col)


#implement the function in df

val cdf= df.columns.foldLeft(df) {(acc,colName)=>

acc.withColumn(colName,repdata(col(colName)))

}

cdf.show()



Comments