How to call print from otherwise in spark scala? - scala

Question

I have dataframe where I'm checking the condition and accordingly I'm performing the operation.I'm using "when and otherwise" function where I'm trying to print failed result in "otherwise" but it is not printing.Any help will be appreciating.
joinDF=
+--------+-------------+----------+--------+-------+
| ID | A | B | C | D |
+--------+-------------+----------+--------+-------+
| 9574| F| 005912| 2016022| 10|
| 9576| F| 005912| 2016022| 21|
| 9578| F| 005912| 2016022| 0|
| 9580| F| 005912| 2016022| 19|
| 9582| F| 005912| 2016022| 89|
+--------+-------------+----------+--------+-------+
joinDF
.withColumn("Validate",when(joinDF("D") =!= 0 ,lit(ture)).otherwise(print(joinDF("ID"))))

Answer 1

It quite simple and very straight-forward :
Seq(("A",1),("B",0)).toDF("key","value")
.withColumn("verdict", when($"value"=!=0, lit("true")).otherwise("false")).show
+---+-----+-------+
|key|value|verdict|
+---+-----+-------+
| A| 1| true|
| B| 0| false|
+---+-----+-------+
You don't need if, else or udfs
With your example :
Seq(("A",1),("B",0)).toDF("ID","D").withColumn("validate",when($"D" =!= 0 ,lit("true")).otherwise($"ID")).show
+---+---+--------+
| ID| D|validate|
+---+---+--------+
| A| 1| true|
| B| 0| B|
+---+---+--------+

Answer 2

If you are trying to print the value if they did not matched than you can create udf as below
val validate = udf((value :String )=>
{
//if if value is not equal to 0 return true else print value and return it
if (value != 0) "true"
else {
print(s"value = ${value}")
value
}
})
//this adds new column named validate and calls the udf validate
joinDF.withColumn("Validate", validate($"ID"))
Hope this helps!