Insert multiple Dataframes into postgres table in a function using scala - postgresql

Question

I have a function :
def PopulatePostgres(df: DataFrame ,df1: DataFrame,df2: DataFrame table: String): Result = {
val result = Try({
df
.write
.format("jdbc")
.mode(SaveMode.Append)
.option("url", config.url)
.option("user", config.username)
.option("password", config.password)
.option("dbtable", table)
.option("driver", "org.postgresql.Driver")
.save()
})
result match {
case Success(_) => Result(s"Created ${table}")
case Failure(problem) => {
log.error(problem.getMessage)
Result(s"Failed to create ${table}")
}
}
}
But,I am not sure how to dump the 3 dataframes one by one into the postgres table .So I need to insert df,df1,df2 all into the postgres table .Can anyone please help me

Answer 1

If you want to store all the data frames into the same table.
val findaldf = df.union(df1).union(df2)
Then you can use your persitance logic.
But all df's are want to store separately
List(df, df1, df2).map(_.write.format("jdbc")
.mode(SaveMode.Append)
.option("url", config.url)
.option("user", config.username)
.option("password", config.password)
.option("dbtable", table)
.option("driver", "org.postgresql.Driver")
.save())

Answer 2

If the schema of all the Dataframes are same (and of course you want them in one table) then you can combine all the them in one and then push the data to postgres else loop it for each dataframe and push the data.
val result = df.unionAll(df1).unionAll(df2)