How to column value split to the row like below input to output via pyspark.

How to column value split to the row like below input to output via pyspark.

tiger analytics
Input:

Name location  Organization

Alex Chennai Org1-Org2-Org3




Output:

Name location Organization

Alex Chennai Org1

Alex Chennai Org2

Alex Chennai Org3








Solve:

dfalex = spark.read.format("csv").option("header","True").load("file:///home/cloudera/alex.txt")

dfalex.show()

dfcon = dfalex.withColumn("organisation",explode(split('organisations','-'))).drop("organisation")

dfcon.show()


Comments