LaVOZs

The World’s Largest Online Community for Developers

'; scala - Reusing vs Cloning a Dataframe in Spark 2.3 for multiple left joins - LavOzs.Com

I am trying to join a master table with multiple Key-Value Dataframe derived from single Dataframe, Spark is reusing the same lineage Dataframe instead of Cloning multiple instances, Due to which only first left join works and others are having NULL values, is there a way to force the Spark to Clone explicitly

Related
How to do left outer join in spark sql?
Spark Scala joining the same data frame and removing bi-directional relationships
How to force DataFrame evaluation in Spark
Join in spark dataframe (scala) based on not null values
Best way to join multiples small tables with a big table in Spark SQL
Partitioning large dataframes for an efficient left join?