I am trying to generate optional parameters in ScalaCheck, without success.
There seems to be no direct mechanism for this. Gen.containerOf[Option, Thing](thingGenerator) fails because it cannot find an implicit Buildable[Thing, Option].
I tried
for {
thing <- Gen.listOfN[Thing](1, thingGenerator)
} yield thing.headOption
But this doesn't work because listOfN produces a list that is always of length N. As a result I always get a Some[Thing]. Similarly, listOf1 does not work, because (a) it doesn't produce empty lists, but also (b) it is inefficient because I can't set a max limit on the number of elements.
How can I generate Option[Thing] that includes Nones?
EDIT: I have found a solution, but it is not succinct. Is there a better way than this?
for {
thing <- for {
qty <- Gen.choose(0,1)
things <- Gen.listOfN[Thing](qty, thingGenerator)
} yield things.headOption
} yield thing
EDIT 2: I generalised this to
def optional[T](g: Gen[T]) =
for (qty <- Gen.choose(0, 1); xs <- Gen.listOfN[T](qty, g)) yield xs.headOption
So I don't have to write it more than once. But surely this is in the library already and I just missed it?
Now you can just use:
Gen.option(yourGen)
You can use pick to randomly choose between a Some and a None generator:
val someThing = thingGenerator.map( Some.apply )
val noThing = Gen.value( None:Option[Thing] )
val optThing = Gen.oneOf( someThing, noThing )
Related
I'm having trouble with a nested for loop and using yield properly.
The problem is if I have two lists of the form List[(Int,Int)]
val ls = (1,5)::(3,2)::(5,3)::Nil
val ls_two = (1,9)::(5,9)::(6,7)::Nil
and now I want to create a third list only combining the key and both second int of all the lists so the result would be
val result = (1,5,9)::(5,3,9)::Nil
I've tried a few variations of something like this which none seem to work
val result = for(i <- ls) {
for(j <- ls_two) {
if(i._1 == j._1) yield (i._1,i._2,j._2)
}
}
I've tried placing the yield at the end of the for loop, it seems to work if i replace yield with println but i'm not sure how to do it with yield.
Also if you have a more functional approach to how to solve this it would be greatly appreciated, thanks!
The recommended approach here is not to "nest" the "loops" at all - but to create a single for-comprehension which uses a "guard":
val result = for {
i <- ls
j <- ls_two if i._1 == j._1
} yield (i._1,i._2,j._2)
I have two functions which return Futures. I'm trying to feed a modified result from first function into the other using a for-yield comprehension.
This approach works:
val schoolFuture = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- schoolStore.getSchool(sid.get) if sid.isDefined
} yield s
However I'm not happy with having the "if" in there, it seems that I should be able to use a map instead.
But when I try with a map:
val schoolFuture: Future[Option[School]] = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- sid.map(schoolStore.getSchool(_))
} yield s
I get a compile error:
[error] found : Option[scala.concurrent.Future[Option[School]]]
[error] required: scala.concurrent.Future[Option[School]]
[error] s <- sid.map(schoolStore.getSchool(_))
I've played around with a few variations, but haven't found anything attractive that works. Can anyone suggest a nicer comprehension and/or explain what's wrong with my 2nd example?
Here is a minimal but complete runnable example with Scala 2.10:
import concurrent.{Future, Promise}
case class User(userId: Int)
case class UserDetails(userId: Int, schoolId: Option[Int])
case class School(schoolId: Int, name: String)
trait Error
class UserStore {
def getUserDetails(userId: Int): Future[Either[Error, UserDetails]] = Promise.successful(Right(UserDetails(1, Some(1)))).future
}
class SchoolStore {
def getSchool(schoolId: Int): Future[Option[School]] = Promise.successful(Option(School(1, "Big School"))).future
}
object Demo {
import concurrent.ExecutionContext.Implicits.global
val userStore = new UserStore
val schoolStore = new SchoolStore
val user = User(1)
val schoolFuture: Future[Option[School]] = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- sid.map(schoolStore.getSchool(_))
} yield s
}
(Edited to give a correct answer!)
The key here is that Future and Option don't compose inside for because there aren't the correct flatMap signatures. As a reminder, for desugars like so:
for ( x0 <- c0; w1 = d1; x1 <- c1 if p1; ... ; xN <- cN) yield f
c0.flatMap{ x0 =>
val w1 = d1
c1.filter(x1 => p1).flatMap{ x1 =>
... cN.map(xN => f) ...
}
}
(where any if statement throws a filter into the chain--I've given just one example--and the equals statements just set variables before the next part of the chain). Since you can only flatMap other Futures, every statement c0, c1, ... except the last had better produce a Future.
Now, getUserDetails and getSchool both produce Futures, but sid is an Option, so we can't put it on the right-hand side of a <-. Unfortunately, there's no clean out-of-the-box way to do this. If o is an option, we can
o.map(Future.successful).getOrElse(Future.failed(new Exception))
to turn an Option into an already-completed Future. So
for {
ud <- userStore.getUserDetails(user.userId) // RHS is a Future[Either[...]]
sid = ud.right.toOption.flatMap(_.schoolId) // RHS is an Option[Int]
fid <- sid.map(Future.successful).getOrElse(Future.failed(new Exception)) // RHS is Future[Int]
s <- schoolStore.getSchool(fid)
} yield s
will do the trick. Is that better than what you've got? Doubtful. But if you
implicit class OptionIsFuture[A](val option: Option[A]) extends AnyVal {
def future = option.map(Future.successful).getOrElse(Future.failed(new Exception))
}
then suddenly the for-comprehension looks reasonable again:
for {
ud <- userStore.getUserDetails(user.userId)
sid <- ud.right.toOption.flatMap(_.schoolId).future
s <- schoolStore.getSchool(sid)
} yield s
Is this the best way to write this code? Probably not; it relies upon converting a None into an exception simply because you don't know what else to do at that point. This is hard to work around because of the design decisions of Future; I'd suggest that your original code (which invokes a filter) is at least as good of a way to do it.
This answer to a similar question about Promise[Option[A]] might help. Just substitute Future for Promise.
I'm inferring the following types for getUserDetails and getSchool from your question:
getUserDetails: UserID => Future[Either[??, UserDetails]]
getSchool: SchoolID => Future[Option[School]]
Since you ignore the failure value from the Either, transforming it to an Option instead, you effectively have two values of type A => Future[Option[B]].
Once you've got a Monad instance for Future (there may be one in scalaz, or you could write your own as in the answer I linked), applying the OptionT transformer to your problem would look something like this:
for {
ud <- optionT(getUserDetails(user.userID) map (_.right.toOption))
sid <- optionT(Future.successful(ud.schoolID))
s <- optionT(getSchool(sid))
} yield s
Note that, to keep the types compatible, ud.schoolID is wrapped in an (already completed) Future.
The result of this for-comprehension would have type OptionT[Future, SchoolID]. You can extract a value of type Future[Option[SchoolID]] with the transformer's run method.
What behavior would you like to occur in the case that the Option[School] is None? Would you like the Future to fail? With what kind of exception? Would you like it to never complete? (That sounds like a bad idea).
Anyways, the if clause in a for-expression desugars to a call to the filter method. The contract on Future#filteris thus:
If the current future contains a value which satisfies the predicate,
the new future will also hold that value. Otherwise, the resulting
future will fail with a NoSuchElementException.
But wait:
scala> None.get
java.util.NoSuchElementException: None.get
As you can see, None.get returns the exact same thing.
Thus, getting rid of the if sid.isDefined should work, and this should return a reasonable result:
val schoolFuture = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- schoolStore.getSchool(sid.get)
} yield s
Keep in mind that the result of schoolFuture can be in instance of scala.util.Failure[NoSuchElementException]. But you haven't described what other behavior you'd like.
We've made small wrapper on Future[Option[T]] which acts like one monad (nobody even checked none of monad laws, but there is map, flatMap, foreach, filter and so on) - MaybeLater. It behaves much more than an async option.
There are a lot of smelly code there, but maybe it will be usefull at least as an example.
BTW: there are a lot of open questions(here for ex.)
It's easier to use https://github.com/qifun/stateless-future or https://github.com/scala/async to do A-Normal-Form transform.
I have a list of possible input Values
val inputValues = List(1,2,3,4,5)
I have a really long to compute function that gives me a result
def reallyLongFunction( input: Int ) : Option[String] = { ..... }
Using scala parallel collections, I can easily do
inputValues.par.map( reallyLongFunction( _ ) )
To get what all the results are, in parallel. The problem is, I don't really want all the results, I only want the FIRST result. As soon as one of my input is a success, I want my output, and want to move on with my life. This did a lot of extra work.
So how do I get the best of both worlds? I want to
Get the first result that returns something from my long function
Stop all my other threads from useless work.
Edit -
I solved it like a dumb java programmer by having
#volatile var done = false;
Which is set and checked inside my reallyLongFunction. This works, but does not feel very scala. Would like a better way to do this....
(Updated: no, it doesn't work, doesn't do the map)
Would it work to do something like:
inputValues.par.find({ v => reallyLongFunction(v); true })
The implementation uses this:
protected[this] class Find[U >: T](pred: T => Boolean, protected[this] val pit: IterableSplitter[T]) extends Accessor[Option[U], Find[U]] {
#volatile var result: Option[U] = None
def leaf(prev: Option[Option[U]]) = { if (!pit.isAborted) result = pit.find(pred); if (result != None) pit.abort }
protected[this] def newSubtask(p: IterableSplitter[T]) = new Find(pred, p)
override def merge(that: Find[U]) = if (this.result == None) result = that.result
}
which looks pretty similar in spirit to your #volatile except you don't have to look at it ;-)
I took interpreted your question in the same way as huynhjl, but if you just want to search and discardNones, you could do something like this to avoid the need to repeat the computation when a suitable outcome is found:
class Computation[A,B](value: A, function: A => B) {
lazy val result = function(value)
}
def f(x: Int) = { // your function here
Thread.sleep(100 - x)
if (x > 5) Some(x * 10)
else None
}
val list = List.range(1, 20) map (i => new Computation(i, f))
val found = list.par find (_.result.isDefined)
//found is Option[Computation[Int,Option[Int]]]
val result = found map (_.result.get)
//result is Option[Int]
However find for parallel collections seems to do a lot of unnecessary work (see this question), so this might not work well, with current versions of Scala at least.
Volatile flags are used in the parallel collections (take a look at the source for find, exists, and forall), so I think your idea is a good one. It's actually better if you can include the flag in the function itself. It kills referential transparency on your function (i.e. for certain inputs your function now sometimes returns None rather than Some), but since you're discarding the stopped computations, this shouldn't matter.
If you're willing to use a non-core library, I think Futures would be a good match for this task. For instance:
Akka's Futures include Futures.firstCompletedOf
Twitter's Futures include Future.select
...both of which appear to enable the functionality you're looking for.
For example suppose I have
for (line <- myData) {
println("}, {")
}
Is there a way to get the last line to print
println("}")
Can you refactor your code to take advantage of built-in mkString?
scala> List(1, 2, 3).mkString("{", "}, {", "}")
res1: String = {1}, {2}, {3}
Before going any further, I'd recommend you avoid println in a for-comprehension. It can sometimes be useful for tracking down a bug that occurs in the middle of a collection, but otherwise leads to code that's harder to refactor and test.
More generally, life usually becomes easier if you can restrict where any sort of side-effect occurs. So instead of:
for (line <- myData) {
println("}, {")
}
You can write:
val lines = for (line <- myData) yield "}, {"
println(lines mkString "\n")
I'm also going to take a guess here that you wanted the content of each line in the output!
val lines = for (line <- myData) yield (line + "}, {")
println(lines mkString "\n")
Though you'd be better off still if you just used mkString directly - that's what it's for!
val lines = myData.mkString("{", "\n}, {", "}")
println(lines)
Note how we're first producing a String, then printing it in a single operation. This approach can easily be split into separate methods and used to implement toString on your class, or to inspect the generated String in tests.
I agree fully with what has been said before about using mkstring, and distinguishing the first iteration rather than the last one. Would you still need to distinguish on the last, scala collections have an init method, which return all elements but the last.
So you can do
for(x <- coll.init) workOnNonLast(x)
workOnLast(coll.last)
(init and last being sort of the opposite of head and tail, which are the first and and all but first). Note however than depending on the structure, they may be costly. On Vector, all of them are fast. On List, while head and tail are basically free, init and last are both linear in the length of the list. headOption and lastOption may help you when the collection may be empty, replacing workOnlast by
for (x <- coll.lastOption) workOnLast(x)
You may take the addString function of the TraversableOncetrait as an example.
def addString(b: StringBuilder, start: String, sep: String, end: String): StringBuilder = {
var first = true
b append start
for (x <- self) {
if (first) {
b append x
first = false
} else {
b append sep
b append x
}
}
b append end
b
}
In your case, the separator is }, { and the end is }
If you don't want to use built-in mkString function, you can make something like
for (line <- lines)
if (line == lines.last) println("last")
else println(line)
UPDATE: As didierd mentioned in comments, this solution is wrong because last value can occurs several times, he provides better solution in his answer.
It is fine for Vectors, because last function takes "effectively constant time" for them, as for Lists, it takes linear time, so you can use pattern matching
#tailrec
def printLines[A](l: List[A]) {
l match {
case Nil =>
case x :: Nil => println("last")
case x :: xs => println(x); printLines(xs)
}
}
Other answers are rightfully pointed to mkString, and for a normal amount of data I would also use that.
However, mkString builds (accumulates) the end-result in-memory through a StringBuilder. This is not always desirable, depending on the amount of data we have.
In this case, if all we want is to "print" we don't need to build the big-result first (and maybe we even want to avoid this).
Consider the implementation of this helper function:
def forEachIsLast[A](iterator: Iterator[A])(operation: (A, Boolean) => Unit): Unit = {
while(iterator.hasNext) {
val element = iterator.next()
val isLast = !iterator.hasNext // if there is no "next", this is the last one
operation(element, isLast)
}
}
It iterates over all elements and invokes operation passing each element in turn, with a boolean value. The value is true if the element passed is the last one.
In your case it could be used like this:
forEachIsLast(myData) { (line, isLast) =>
if(isLast)
println("}")
else
println("}, {")
}
We have the following advantages here:
It operates on each element, one by one, without necessarily accumulating the result in memory (unless you want to).
Because it does not need to load the whole collection into memory to check its size, it's enough to ask the Iterator if it's exhausted or not. You could read data from a big file, or from the network, etc.
Suppose I have two Options and, if both are Some, execute one code path, and if note, execute another. I'd like to do something like
for (x <- xMaybe; y <- yMaybe) {
// do something
}
else {
// either x or y were None, handle this
}
Outside of if statements or pattern matching (which might not scale if I had more than two options), is there a better way of handling this?
Very close to your syntax proposal by using yield to wrap the for output in an Option:
val result = {
for (x <- xMaybe; y <- yMaybe) yield {
// do something
}
} getOrElse {
// either x or y were None, handle this
}
The getOrElse block is executed only if one or both options are None.
You could pattern match both Options at the same time:
(xMaybe, yMaybe) match {
case (Some(x), Some(y)) => "x and y are there"
case _ => "x and/or y were None"
}
The traverse function in Scalaz generalises your problem here. It takes two arguments:
T[F[A]]
A => F[B]
and returns F[T[B]]. The T is any traversable data structure such as List and the F is any applicative functor such as Option. Therefore, to specialise, your desired function has this type:
List[Option[A]] => (A => Option[B]) => Option[List[B]]
So put all your Option values in a List
val z = List(xMaybe, yMaybe)
Construct the function got however you want to collection the results:
val f: X => Option[Y] = ...
and call traverse
val r = z traverse f
This programming patterns occurs very often. It has a paper that talks all about it, The Essence of the Iterator Pattern.
note: I just wanted to fix the URL but the CLEVER edit help tells me I need to change at least 6 characters so I include this useful link too (scala examples):
http://etorreborre.blogspot.com/2011/06/essence-of-iterator-pattern.html
Why would something like this not work?
val opts = List[Option[Int]](Some(1), None, Some(2))
if (opts contains None) {
// Has a None
} else {
// Launch the missiles
val values = opts.map(_.get) // We know that there is no None in the list so get will not throw
}
If you don't know the number of values you are dealing with, then Tony's answer is the best. If you do know the number of values you are dealing with then I would suggest using an applicative functor.
((xMaybe |#| yMaybe) { (x, y) => /* do something */ }).getOrElse(/* something else */)
You said you want the solution to be scalable:
val optional = List(Some(4), Some(3), None)
if(optional forall {_.isDefined}) {
//All defined
} else {
//At least one not defined
}
EDIT: Just saw that Emil Ivanov's solution is a bit more elegant.
Starting Scala 2.13, we can alternatively use Option#zip which concatenates two options to Some tuple of their values if both options are defined or else None:
opt1 zip opt2 match {
case Some((x, y)) => "x and y are there"
case None => "x and/or y were None"
}
Or with Option#fold:
(opt1 zip opt2).fold("x and/or y were None"){ case (x, y) => "x and y are there" }
For scaling to many options, try something along these lines:
def runIfAllSome[A](func:(A)=>Unit, opts:Option[A]*) = {
if(opts.find((o)=>o==None) == None) for(opt<-opts) func(opt.get)
}
With this, you can do:
scala> def fun(i:Int) = println(i)
fun: (i: Int)Unit
scala> runIfAllSome(fun, Some(1), Some(2))
1
2
scala> runIfAllSome(fun, None, Some(1))
scala>
I think the key point here is to think in term of types as what you want to do. As I understand it you want to iterate over a list of Option pairs and then do something based on a certain condition. So the interesting bit of your question would be , what would the return type look like you would except? I think it would look something like this: Either[List[Option], List [Option,Option]] . on the error side (left) you would accumulate the option which was paired with a None (and was left alone so to speak) . On the right side you sum the non empty options which represent your successful values. So we would just need a function which does exactly that. Validate each pair and accumulate it according to it's result( success - failure) . I hope this helps , if not please explain in more detail your usecase. Some links to implement what I described : http://applicative-errors-scala.googlecode.com/svn/artifacts/0.6/pdf/index.pdf and : http://blog.tmorris.net/automated-validation-with-applicatives-and-semigroups-for-sanjiv/