Read with spark ORC file with corrupted maps - scala

Question

I have orc files in my hdfs. One of the fields is Map(String, String). Somehow there are some rows with value Map(null,null) in this field. null in the map keys is a critical error for java. So, when I'm trying to access this field, I got NullPointer exeption.
I want to read these files and change this fields to emty map.
I tried to do it this way:
val df = spark.read.format("orc").load("/tmp/bad_orc")
def func(s: org.apache.spark.sql.Row): String = {
try
{
if ( s(14) == null ) // the 14'th column is the column with Map(String,String) type
{
return "Ok"
}
else
{
return "Zero"
}
}
catch
{
case x: Exception => return "Erro"
}
}
df.rdd.map(func).take(20)
I got this exception, when I run this script.
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 417.0 failed 4 times, most recent failure: Lost task 0.3 in stage 417.0 (TID 97094, srvg1076.local.odkl.ru, executor 86): java.lang.NullPointerException
at java.util.TreeMap.compare(TreeMap.java:1294)
at java.util.TreeMap.put(TreeMap.java:538)
at org.apache.orc.mapred.OrcMapredRecordReader.nextMap(OrcMapredRecordReader.java:507)
at org.apache.orc.mapred.OrcMapredRecordReader.nextValue(OrcMapredRecordReader.java:554)
at org.apache.orc.mapreduce.OrcMapreduceRecordReader.nextKeyValue(OrcMapreduceRecordReader.java:104)
at org.apache.spark.sql.execution.datasources.RecordReaderIterator.hasNext(RecordReaderIterator.scala:39)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.hasNext(FileScanRDD.scala:101)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$11$$anon$1.hasNext(WholeStageCodegenExec.scala:619)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
And when I'm trying to acces any other column in this orc - everything is Ok.
How catch this exception and how fix these files? Help me, please