I have a method that return (String) => Try[(Long, String)] type and I want to get (Long, String). Any suggestion?
I thought map/flatMap will help but looks like they doesn't.
Update
def someMethod():(Long, String) = {
val result: (String) => Try[(Long, String)] = someOperation()
//Need to get (Long, String) from result
}
There are several options
val exceptional: Try[(Long, String)] = ???
val default: (Long, String) = (0, "")
Providing fallback value
exceptional.getOrElse(default)
handling exception and then safely get
exceptional.recover { case exception => default }.get
or using pattern matching
exceptional match {
case Success(v) => v
case Failure(exception) => default
}
Related
I have the following overloaded method which input can be a Option[String] or Option[Seq[String]]:
def parse_emails(email: => Option[String]) : Seq[String] = {
email match {
case Some(e : String) if e.isEmpty() => null
case Some(e : String) => Seq(e)
case _ => null
}
}
def parse_emails(email: Option[Seq[String]]) : Seq[String] = {
email match {
case Some(e : Seq[String]) if e.isEmpty() => null
case Some(e : Seq[String]) => e
case _ => null
}
}
I want to use this method from Spark, so I tried to wrap them as a udf:
def parse_emails_udf = udf(parse_emails _)
But I am getting the following error:
error: ambiguous reference to overloaded definition,
both method parse_emails of type (email: Option[Seq[String]])Seq[String]
and method parse_emails of type (email: => Option[String])Seq[String]
match expected type ?
def parse_emails_udf = udf(parse_emails _)
Is it possible to define a udf which could wrap both alternative?
Or could it be possible to create two udfs with same name each pointing to one of the overloaded options? I tried below approach, but throws another error:
def parse_emails_udf = udf(parse_emails _ : Option[Seq[String]])
error: type mismatch;
found : (email: Option[Seq[String]])Seq[String] <and> (email: => Option[String])Seq[String]
required: Option[Seq[String]]
def parse_emails_udf = udf(parse_emails _ : Option[Seq[String]])
Option[String] and Option[Seq[String]] have the same erasure Option, so even if Spark supported udf overloading it wouldn't work.
What you can do is create one function that accepts anything, then match on the argument and handle the different cases:
def parseEmails(arg: Option[AnyRef]) = arg match {
case Some(x) =>
x match {
case str: String =>
??? // todo
case s: Seq[String] =>
??? // todo
case _ =>
throw new IllegalArgumentException()
}
case None =>
??? // todo
}
I have an iterable of arrays that I am trying to turn into case classes, and I'm mapping over them to do so. In the event of an array being non-convertable to a case class, I want to log a warning and proceed with the mapping. However, when I implement the warning, the return type changes from Iterable[MyCaseClass] to Iterable[Any] which is not what I want. E.g.:
case class MyCaseClass(s1: String, s2: String)
object MyCaseClass {
def apply(sa: Array[String]) = new MyCaseClass(sa(0), sa(1))
}
val arrayIterable: Iterable[Array[String]] = Iterable(Array("a", "b"), Array("a", "b", "c"))
def badReturnType(): Iterable[Any] = { // Iterable[Any] is undesireable
arrayIterable map {
case sa: Array[String] if sa.length == 2 => MyCaseClass(sa)
case _ => println("something bad happened!") // but warnings are good
}
}
def desiredReturnType(): Iterable[MyCaseClass] = { // Iterable[MyCaseClass] is desireable
arrayIterable map {
case sa: Array[String] if sa.length == 2 => MyCaseClass(sa)
// but no warnings if things go wrong!
}
}
I want to write a function that meets the following criteria:
maps over the Iterable, converting each element to a MyCaseClass
log warnings when I get an array that cant be converted to a MyCaseClass
after logging the warning, the array passed into the match criteria is ignored/discarded
the return type should be Iterable[MyCaseClass].
How can I meet these conditions?
Consider using List instead of Array, and try wrapping in Option in combination with flatMap
l flatMap {
case e if e.length == 2 => Some(MyCaseClass(e))
case e => println(s"$e is wrong length"); None
}
Another approach is partitionMap
val (lefts, rights) = l.partitionMap {
case e if e.size == 2 => Right(MyCaseClass(e))
case e => Left(s"$e is wrong length")
}
lefts.foreach(println)
rights
You can do something like this:
final case class MyCaseClass(s1: String, s2: String)
def parse(input: Array[String]): Either[String, MyCaseClass] = input match {
case Array(s1, s2) => Right(MyCaseClass(s1, s2))
case _ => Left(s"Bad input: ${input.mkString("[", ", ", "]")}")
}
def logErrors(validated: Either[String, _]): Unit = validated match {
case Left(error) => println(error)
case Right(_) => ()
}
def validateData(data: IterableOnce[Array[String]]): List[MyCaseClass] =
data
.iterator
.map(parse)
.tapEach(logErrors)
.collect {
case Right(value) => value
}.toList
Which you can use like this:
val arrayIterable = Iterable(Array("a", "b"), Array("a", "b", "c"))
validateData(arrayIterable)
// Bad input: [a, b, c]
// res14: List[MyCaseClass] = List(MyCaseClass("a", "b"))
I have the following function definition:
private def extractUrl: String => (String, String)
= url =>
url
.split("/")
.toList
.filter(_.startsWith("localhost"))
.flatMap(e => e.split(":").toList)
.foldLeft[(String, String)](("", "")) { (acc, x) =>
acc match {
case ("", "") => (x, "")
case (a, "") => (a, x)
case z => z
}
}
the question is, is there another way to define an empty Tuple instead of ("", "")?
Empty tuple
("", "") is a tuple of empty strings with type (String, String).
Empty is unclear in that context, could be (None, None) or even (null, null) (bad)
You seem to use "" to represents a value that is not present. Try using None and Some[String], both sub types of Option[String], to indicate that a value is not present.
Analysis and comments
Potentially, your method seems not to do what is intended. (execute code below)
Think about using already present functions / methods / libraries for manipulating URLs (also see example below)
Think about using Option
object Fun {
import java.net.URL
def main(args: Array[String]): Unit = {
val url1 = "http://localhost:4000/a/b/c?x=1&y=2#asdf"
val url2 = "http://example.com:4000/a/localhostb/c?x=1&y=2#asdf"
val urls = List(url1, url2)
// your approach
println("Your approach")
urls.map( urlString => extractUrl(urlString ))
.foreach(println)
println("New approach")
urls.map(x => extractUrl2(x))
.filter( x => x.host.startsWith("localhost") )
.foreach(println)
}
case class HostPort(host: String, port: Option[String])
def extractUrl2: String => HostPort = urlString => {
val url = new URL(urlString)
HostPort(url.getHost,
url.getPort match {
case -1 => None
case i => Some(i.toString)
})
}
def extractUrl: String => (String, String) = url =>
url
.split("/")
.toList
.filter(_.startsWith("localhost"))
.flatMap(e => e.split(":").toList)
.foldLeft[(String, String)](("", "")) { (acc, x) =>
acc match {
case ("", "") => (x, "")
case (a, "") => (a, x)
case z => z
}
}
}
yields
Your approach
(localhost,4000)
(localhostb,)
New approach
HostPort(localhost,Some(4000))
I don't think it is possible to define an empty Tuple.
I tried to use (->) but that resolves to a Tuple2.type and not a Tuple2.
If the values of your Tuple are optional, use the type system to express that:
(Option[String], Option[String])
Better yet, you could define a case class for your data structure:
case class HostAndPort(host: Option[String], port: Option[String])
This would provide default values for each type within the tuple:
let my_tuple: (String, usize) = Default::default();
I'm using spray-json and I need to parse the given request body (PATCH, POST), request body attributes can have following possibilities represented by Either[Unit.type, Option[A]]
value Not given Left[Unit.type]
value=null Null Right[None]
value=XXX Some value is provided Right[Some(value)]
Using the above possibilities I need to create a entity from the request body. While parsing I need to validate each field with some business logic (String length, integer range ...).
I have a following function for the business logic validation.
def validateValue[T](fieldName: String,
maybeValue: Try[T],
businessValidation: T => Boolean): Option[T] = {
maybeValue match {
case Success(value) if businessValidation(value) => Some(value)
case _ => None
}
}
Similarly another function readFieldWithValidation, here I will be parsing each attribute based on the input type and apply the business validation.
def readFieldWithValidation[S, T](fields: Map[String, JsValue], fieldName: String, businessValidation: T => Boolean)(
parse: S => T
): Option[T] = {
fields.get(fieldName) match {
case None => None
case Some(jsValue) =>
jsValue match {
case jsString: JsString =>
validateValue(fieldName, Try(parse(jsString.value)), businessValidation)
case JsNumber(jsNumber) =>
validateValue(fieldName, Try(parse(jsNumber.intValue)), businessValidation)
case _ => None
}
}
}
I have S ( Source ) and T ( Target ) which is used for given a JsValue returns T type. Here I only care about JsString and JsNumber.
The above lines of code is giving type mismatch error,
<console>:112: error: type mismatch;
found : jsString.value.type (with underlying type String)
required: S
validateValue(fieldName, Try(parse(jsString.value)), businessValidation)
^
<console>:114: error: type mismatch;
found : Int
required: S
validateValue(fieldName, Try(parse(jsNumber.intValue)), businessValidation)
Can someone help me how to overcome this error?
This is how I can use above function
val attributes = Map("String" -> JsString("ThisIsString"), "Int" -> JsNumber(23))
def stringLengthConstraint(min: Int, max: Int)(value: String) = value.length > min && value.length < max
readFieldWithValidation[JsString, String](attributes, "String", stringLengthConstraint(1, 10))(_.toString)
Your example is still not quite clear because it does not show the role of parse and actually looks contradictory to the other code: particularly you specify the generic parameter S as JsString in readFieldWithValidation[JsString, String] but given current (borken) readFieldWithValidation implementation your parse argument is probably expected to be of type String => String because jsString.value is String.
Anyway here is a piece of code that seem to implement something that is hopefully sufficiently close to what you want:
trait JsValueExtractor[T] {
def getValue(jsValue: JsValue): Option[T]
}
object JsValueExtractor {
implicit val decimalExtractor = new JsValueExtractor[BigDecimal] {
override def getValue(jsValue: JsValue) = jsValue match {
case JsNumber(jsNumber) => Some(jsNumber)
case _ => None
}
}
implicit val intExtractor = new JsValueExtractor[Int] {
override def getValue(jsValue: JsValue) = jsValue match {
case JsNumber(jsNumber) => Some(jsNumber.intValue)
case _ => None
}
}
implicit val doubleExtractor = new JsValueExtractor[Double] {
override def getValue(jsValue: JsValue) = jsValue match {
case JsNumber(jsNumber) => Some(jsNumber.doubleValue)
case _ => None
}
}
implicit val stringExtractor = new JsValueExtractor[String] {
override def getValue(jsValue: JsValue) = jsValue match {
case JsString(string) => Some(string)
case _ => None
}
}
}
def readFieldWithValidation[S, T](fields: Map[String, JsValue], fieldName: String, businessValidation: T => Boolean)(parse: S => T)(implicit valueExtractor: JsValueExtractor[S]) = {
fields.get(fieldName)
.flatMap(jsValue => valueExtractor.getValue(jsValue))
.flatMap(rawValue => Try(parse(rawValue)).toOption)
.filter(businessValidation)
}
and usage example:
def test(): Unit = {
val attributes = Map("String" -> JsString("ThisIsString"), "Int" -> JsNumber(23))
def stringLengthConstraint(min: Int, max: Int)(value: String) = value.length > min && value.length < max
val value = readFieldWithValidation[String, String](attributes, "String", stringLengthConstraint(1, 10))(identity)
println(value)
}
Your current code uses Option[T] as your return type. If I were using a code like this I'd probably added some error logging and/or handling for a case where the code contains a bug and attributes do contain a value for key fieldName but of some different, unexpected type (like JsNumber instead of JsString).
Update
It is not clear from your comment whether you are satisfied with my original answer or want to add some error handling. If you want to report the type mismatch errors, and since you are using cats, something like ValidatedNel is an obvious choice:
type ValidationResult[A] = ValidatedNel[String, A]
trait JsValueExtractor[T] {
def getValue(jsValue: JsValue, fieldName: String): ValidationResult[T]
}
object JsValueExtractor {
implicit val decimalExtractor = new JsValueExtractor[BigDecimal] {
override def getValue(jsValue: JsValue, fieldName: String): ValidationResult[BigDecimal] = jsValue match {
case JsNumber(jsNumber) => jsNumber.validNel
case _ => s"Field '$fieldName' is expected to be decimal".invalidNel
}
}
implicit val intExtractor = new JsValueExtractor[Int] {
override def getValue(jsValue: JsValue, fieldName: String): ValidationResult[Int] = jsValue match {
case JsNumber(jsNumber) => Try(jsNumber.toIntExact) match {
case scala.util.Success(intValue) => intValue.validNel
case scala.util.Failure(e) => s"Field $fieldName is expected to be int".invalidNel
}
case _ => s"Field '$fieldName' is expected to be int".invalidNel
}
}
implicit val doubleExtractor = new JsValueExtractor[Double] {
override def getValue(jsValue: JsValue, fieldName: String): ValidationResult[Double] = jsValue match {
case JsNumber(jsNumber) => jsNumber.doubleValue.validNel
case _ => s"Field '$fieldName' is expected to be double".invalidNel
}
}
implicit val stringExtractor = new JsValueExtractor[String] {
override def getValue(jsValue: JsValue, fieldName: String): ValidationResult[String] = jsValue match {
case JsString(string) => string.validNel
case _ => s"Field '$fieldName' is expected to be string".invalidNel
}
}
}
def readFieldWithValidation[S, T](fields: Map[String, JsValue], fieldName: String, businessValidation: T => Boolean)
(parse: S => T)(implicit valueExtractor: JsValueExtractor[S]): ValidationResult[T] = {
fields.get(fieldName) match {
case None => s"Field '$fieldName' is required".invalidNel
case Some(jsValue) => valueExtractor.getValue(jsValue, fieldName)
.andThen(rawValue => Try(parse(rawValue).validNel).getOrElse("".invalidNel))
.andThen(parsedValue => if (businessValidation(parsedValue)) parsedValue.validNel else s"Business validation for field '$fieldName' has failed".invalidNel)
}
}
And the test example remains the same. Probably in your real code you want to use something more specific than just String for errors but that's up to you.
I am trying to parse a csv row here and each field can be a different type. To handle the error accumulation I am using Either[String, B] where the String is an error message and B is the value. The issue here is that B can be different types, Option[Int], String, Array[String], resulting in my Map being type (String, Either[String,java.io.Serializable]) effectively making the Map unreusable. Is there a way (I'm definitely sure there is) to more elegantly accumulate errors while also reusing those values to populate properties on an object?
override def parseCsv(implicit fields: Map[String, String]): Either[String, User] = {
val parsedValues = Map(Headers.Id -> getFieldAsString(Headers.Id),
Headers.FirstName -> getFieldAsString(Headers.FirstName),
Headers.LastName -> getFieldAsString(Headers.LastName),
Headers.Email -> getFieldAsString(Headers.Email),
Headers.TimeZone -> getFieldAsString(Headers.TimeZone),
Headers.Region -> getOption(Headers.Region),
Headers.Phone -> getOption(Headers.Phone),
Headers.ProfileImage -> getFieldAsString(Headers.ProfileImage),
Headers.Roles -> getFieldAsArray(Headers.Roles))
val errors = parsedValues.collect { case (key, Left(errors)) => errors }
if (!errors.isEmpty) Left(errors.mkString(", "))
else {
val user = new User
user.id = getFieldAsString(Headers.Id).right.get
user.firstName = getFieldAsString(Headers.FirstName).right.get
user.lastName = getFieldAsString(Headers.LastName).right.get
user.email = getFieldAsString(Headers.Email).right.get
user.timeZone = getFieldAsString(Headers.TimeZone).right.get
user.phoneNumber = (for {
region <- getOption(Headers.Region).right.get
phone <- getOption(Headers.Phone).right.get
_ = validatePhoneNumber(phone, region)
} yield {
new PhoneNumber(region, phone)
}).orNull
user.profileImageUrl = getFieldAsString(Headers.ProfileImage).right.get
user.roles = getFieldAsArray(Headers.Roles).right.get
Right(user)
}
}
Create case classes for all types of Bs. These case classes must extend some common trait. While populating the user object just pattern match and retrieve values.
sealed trait Result {
val paramName: String
}
case class OptionInt(override val paramName: String, value: Option[Int]) extends Result
case class ArrayString(override val paramName: String, value: Array[String]) extends Result
case class StringValue(override val paramName: String, value: String) extends Result
now the final type would be like Either[String, Result]
after parsing the whole file create a List[Result]
If you are expecting age as Option[Int] and firstName as String then do this
list.foreach { result =>
result match {
case Option("age", value) => userId.age = value.getOrElse(defaultAge)
case StringValue("firstName", value) => userId.firstName = value
case StringValue("email", value) => userId.email = value
case _ => //do nothing
}
}