Mocking functions used inside spark udfs - scala

Question

I had a user defined function as follows:
def myFunc(df: DataFrame, column_name: String, func:(String)=>String = myFunc) = {
val my_udf = udf{ (columnValue: String) => func(columnValue) }
df.withColumn("newColumn", my_udf(df(column_name)))
}
Now, while writing test cases, I wanted to mock myFunc. I'm using Mockito for testing.
So I created a mock function with
var fmock = mock[(String)=>String]
when(fmock(any[String])).thenReturn("default_string_write_value")
However, when I try to pass fmock, I get the following exception: java.io.NotSerializableException.
So I tried to make fmock serializable by declaring it as follows:
var fmock = mock[(String)=>String](Mockito.withSettings().serializable())
Now when I call a collect on the dataframe returned by myFunc, I get the following exception:
java.lang.ClassCastException: cannot assign instance of scala.collection.immutable.List$SerializationProxy to field org.apache.spark.rdd.RDD.org$apache$spark$rdd$RDD$$dependencies_ of type scala.collection.Seq in instance of org.apache.spark.rdd.MapPartitionsRDD
This error is obtained when I am trying to perform the collect operation.
Also, a weird thing is happening. The test case passes in IntelliJ (that's the IDE I'm using), but fails when I run it with Maven.
The test case is as follows (assume I already have a dataframe-TESTDF in place with columns-COL1,COL2,COL3):
val fmock = mock[(String)=>String](Mockito.withSettings().serializable())
when(fmock(any[String])).thenReturn("default_string_write_value")
val ansDf = myFunc(TESTDF, COL1, fmock)
var rowValueArray = ansDf.select("newColumn").collect.map(x => x(0).asInstanceOf[String])
assert(rowValueArray(0)=="default_string_write_value")
Note that I'm using Spark versions 2.3.3 and scala 2.11.8. The application is running fine, the issue only comes in with test cases (that too on maven).