Left join with two dataframe what you data you found

altimetric

Input:

Dept_table

+----+---+

| Dno|loc|

+----+---+

|  10|Hyd|

|null|Hyd|

+----+---+

Employee_table

+---+----+

|eno| dno|

+---+----+

|101|  10|

|102|  10|

|103|null|

+---+----+

Solve:

dept_table_data = [(10,"Hyd"),(None,"Hyd")]

 dept_schema = StructType([StructField("Dno",IntegerType(),True),StructField("loc",StringType(),True)])

 dept_tables = spark.createDataFrame(dept_table_data,dept_schema)

 dept_tables.show()

+----+---+

| Dno|loc|

+----+---+

|  10|Hyd|

|null|Hyd|

+----+---+

 

 emp_table_data = [(101,10),(102,10),(103,None)]

 emp_schema = StructType([StructField("eno",IntegerType(),True),StructField("dno",IntegerType(),True)])

 emp_tables = spark.createDataFrame(emp_table_data,emp_schema)

 emp_tables.show()

+---+----+

|eno| dno|

+---+----+

|101|  10|

|102|  10|

|103|null|

+---+----+

 emp_join = dept_tables.join(emp_tables,emp_tables.dno==dept_tables.Dno,"left").drop(dept_tables.Dno)

 emp_join.show()

+---+----+----+

|loc| eno| dno|

+---+----+----+

|Hyd|null|null|

|Hyd| 101|  10|

|Hyd| 102|  10|

+---+----+----+