quite new to scala, hoping to get some help, if I have a method like this, how to test this prepareCappuccino method? Is there something like mokito.spy one this inner method? Thanks
http://danielwestheide.com/blog/2013/01/09/the-neophytes-guide-to-scala-part-8-welcome-to-the-future.html
def prepareCappuccino(): Future[Cappuccino] = {
def grind
def heatWater
def fronthMilk
def brew
// implementation of these methods above within this prepareCappuccino method
for {
ground <- grind("arabica beans")
water <- heatWater(Water(20))
foam <- frothMilk("milk")
espresso <- brew(ground, water)
} yield combine(espresso, foam)
}
As you have hardcoded the actual values you could simply test the expected ouput.
import scala.concurrent.ExecutionContext.Implicits.global
val expected = Cappuccino(...)
val f = prepareCappuccino()
f.map(c => assert(c == expected))
Have a look at ScalaTest, Specs2 and ScalaCheck which are the established testing frameworks for scala. ScalaCheck gives you property based testing and choosing between the first two is just a matter of taste. Just pick one that you like more.
Handling of futures can be done via blocking (discouraged) or async testing. Here is a link to the ScalaTest docs regarding it: http://www.scalatest.org/user_guide/async_testing
Please note that in "real code" you should avoid structuring your code in a way that complicates testing.
Related
I understand that generally speaking there is a lot to say about deciding what one wants to model as effect This discussion is introduce in Functional programming in Scala on the chapter on IO.
Nonethless, I have not finished the chapter, i was just browsing it end to end before takling it together with Cats IO.
In the mean time, I have a bit of a situation for some code I need to deliver soon at work.
It relies on a Java Library that is just all about mutation. That library was started a long time ago and for legacy reason i don't see them changing.
Anyway, long story short. Is actually modeling any mutating function as IO a viable way to encapsulate a mutating java library ?
Edit1 (at request I add a snippet)
Readying into a model, mutate the model rather than creating a new one. I would contrast jena to gremlin for instance, a functional library over graph data.
def loadModel(paths: String*): Model =
paths.foldLeft(ModelFactory.createOntologyModel(new OntModelSpec(OntModelSpec.OWL_MEM)).asInstanceOf[Model]) {
case (model, path) ⇒
val input = getClass.getClassLoader.getResourceAsStream(path)
val lang = RDFLanguages.filenameToLang(path).getName
model.read(input, "", lang)
}
That was my scala code, but the java api as documented in the website look like this.
// create the resource
Resource r = model.createResource();
// add the property
r.addProperty(RDFS.label, model.createLiteral("chat", "en"))
.addProperty(RDFS.label, model.createLiteral("chat", "fr"))
.addProperty(RDFS.label, model.createLiteral("<em>chat</em>", true));
// write out the Model
model.write(system.out);
// create a bag
Bag smiths = model.createBag();
// select all the resources with a VCARD.FN property
// whose value ends with "Smith"
StmtIterator iter = model.listStatements(
new SimpleSelector(null, VCARD.FN, (RDFNode) null) {
public boolean selects(Statement s) {
return s.getString().endsWith("Smith");
}
});
// add the Smith's to the bag
while (iter.hasNext()) {
smiths.add(iter.nextStatement().getSubject());
}
So, there are three solutions to this problem.
1. Simple and dirty
If all the usage of the impure API is contained in single / small part of the code base, you may just "cheat" and do something like:
def useBadJavaAPI(args): IO[Foo] = IO {
// Everything inside this block can be imperative and mutable.
}
I said "cheat" because the idea of IO is composition, and a big IO chunk is not really composition. But, sometimes you only want to encapsulate that legacy part and do not care about it.
2. Towards composition.
Basically, the same as above but dropping some flatMaps in the middle:
// Instead of:
def useBadJavaAPI(args): IO[Foo] = IO {
val a = createMutableThing()
mutableThing.add(args)
val b = a.bar()
b.computeFoo()
}
// You do something like this:
def useBadJavaAPI(args): IO[Foo] =
for {
a <- IO(createMutableThing())
_ <- IO(mutableThing.add(args))
b <- IO(a.bar())
result <- IO(b.computeFoo())
} yield result
There are a couple of reasons for doing this:
Because the imperative / mutable API is not contained in a single method / class but in a couple of them. And the encapsulation of small steps in IO is helping you to reason about it.
Because you want to slowly migrate the code to something better.
Because you want to feel better with yourself :p
3. Wrap it in a pure interface
This is basically the same that many third party libraries (e.g. Doobie, fs2-blobstore, neotypes) do. Wrapping a Java library on a pure interface.
Note that as such, the amount of work that has to be done is way more than the previous two solutions. As such, this is worth it if the mutable API is "infecting" many places of your codebase, or worse in multiple projects; if so then it makes sense to do this and publish is as an independent module.
(it may also be worth to publish that module as an open-source library, you may end up helping other people and receive help from other people as well)
Since this is a bigger task is not easy to just provide a complete answer of all you would have to do, it may help to see how those libraries are implemented and ask more questions either here or in the gitter channels.
But, I can give you a quick snippet of how it would look like:
// First define a pure interface of the operations you want to provide
trait PureModel[F[_]] { // You may forget about the abstract F and just use IO instead.
def op1: F[Int]
def op2(data: List[String]): F[Unit]
}
// Then in the companion object you define factories.
object PureModel {
// If the underlying java object has a close or release action,
// use a Resource[F, PureModel[F]] instead.
def apply[F[_]](args)(implicit F: Sync[F]): F[PureModel[F]] = ???
}
Now, how to create the implementation is the tricky part.
Maybe you can use something like Sync to initialize the mutable state.
def apply[F[_]](args)(implicit F: Sync[F]): F[PureModel[F]] =
F.delay(createMutableState()).map { mutableThing =>
new PureModel[F] {
override def op1: F[Int] = F.delay(mutableThing.foo())
override def op2(data: List[String]): F[Unit] = F.delay(mutableThing.bar(data))
}
}
Tried this to create a seq from file:
def getFileAsList(bufferedReader: BufferedReader): Seq[String] ={
import resource._
for(source <- managed(bufferedReader)){
for(line<-source.lines())
yield line
}
}
I don't think you use Scala-ARM in a way it was designed to be used. The thing is that unless you use Imperative style i.e. consume your managed resource in place, you use Monadic style so what you get is result wrapped into a ExtractableManagedResource which is a delayed (lazy) computation rather than an immediate result. So this is not a direct substitute for Java try-with-resource construct. Monadic style is more useful if you have a method that wants to return some lazy resource that is also happens to be managed i.e. requires some kind of explicit close after usage. But this means that the managed resource is created inside the method rather than passed from the outside as in your case.
Still you probably can achieve something similar to what you want with a construction like
def getFileAsList(bufferedReader: BufferedReader): java.util.stream.Stream[String] = {
import resource._
val managedWrapper = for (source <- managed(bufferedReader))
yield for (line <- source.lines())
yield line
managedWrapper.tried.get
}
The tried method converts ExtractableManagedResource into a Try and get on that will either get you the result or (re-)throw the exception that happened during result calculation.
Please also note, that java.util.Stream is a beast quite different from scala.collection.Seq or scala.collection.Stream. If you want get Scala-specific Stream you should use some Scala-specific code such as
def getFileAsList(bufferedReader: BufferedReader): scala.collection.immutable.Stream[String] = {
import resource._
val managedWrapper = for (source <- managed(bufferedReader))
yield Stream.continually(source.readLine()).takeWhile(_ != null)
managedWrapper.tried.get
}
I'm learning about the Free monad in Scala, and I've put together a simple example of algebra that I can lift into a Free monad using cats.
Here's my algebra
sealed trait ConsultationOp[A]
object consultation {
case class Create(c: Consultation) extends ConsultationOp[Unit]
case class Get(s: ConsultationId) extends ConsultationOp[Option[Consultation]]
}
And I'm able to use it like
def app = for {
c <- consultation.Create(Consultation("123", "A consultation"))
_ <- consultation.Get(c._id)
} yield ()
def interpreters = ConsultationInterpreter or UserInterpreter
app.foldMap(interpreters)
Where the lifting from ConsultationOp to Free is performed implicitly.
(there's a lot of details missing, the full working implementation is here: https://github.com/gabro/free-api)
So far so good, but what if I need to extract the optional value returned by consultation.Get.
The first thing that comes to mind is a monad transformer, i.e. something like
def app = for {
c <- consultation.Create(Consultation("123", "A consultation")).liftM[OptionT]
d <- OptionT(consultation.Get(c._id))
_ <- doSomethingAConsultation(d)
} yield ()
but it looks ugly, and it doesn't feel right.
What's the glorified way - if any - of stacking monadic effects when using a Free monad?
The common way I see recurring in these cases is to use traverse, so you could change your code along the lines of:
import cats.syntax.traverse._
import cats.instances.option._
// ...
def app = for {
c <- consultation.Create(Consultation("123", "A consultation"))
d <- consultation.Get(c._id)
_ <- d.traverseU(doSomethingAConsultation(_))
} yield ()
Which, imho, is much cleaner than the monad transformer alternative.
Note that you could need some other import and slightly modify the code, I didn't try it, but the concept is: use traverse.
Experimenting with concurrent execution I was wondering how to actually test it.
The execution flow is of a side-effect nature and futures are created to wrap independent executions/processing.
Been searching for some good examples on how to properly unit test the following scenarios (foo and bar are the methods I wish to test):
scenario #1
def foo : Unit = {
Future { doSomething }
Future { doSomethingElse }
}
private def doSomething : Unit = serviceCall1
private def doSomethingElse : Unit = serviceCall2
Scenario motivation
foo immediately returns but invokes 2 futures which perform separate tasks (e.g. save analytics and store record to DB). These service calls can be mocked, but what I'm trying to test is that both these services are called once I wrap them in Futures
scenario #2
def bar : Unit = {
val futureX = doAsyncX
val futureY = doAsyncY
for {
x <- futureX
y <- futureY
} yield {
noOp(x, y)
}
}
Scenario motivation
Start with long running computations that can be executed concurrently (e.g. get the number of total visitors and get the frequently used User-Agent header to our web site). Combine the result in some other operation (which in this case Unit method that simply throws the values)
Note I'm familiar with actors and testing actors, but given the above code I wonder what should be the most suitable approach (refactoring included)
EDIT What I'm doing at the moment
implicit value context = ExecutionContext.fromExecutor(testExecutor)
def testExecutor = {
new Executor {
def execute(runnable : Runnable) = runnable.run
}
}
This ExecutionContext implementation will not run the Future as a separate thread and the entire execution will be done in sequence. This kinda feels like a hack but based on Electric Monk answer, it seems like the other solution is more of the same.
One solution would be to use a DeterministicExecutor. Not a scalaesque solution, but should so the trick.
If you are using ScalaTest, take a look at: http://doc.scalatest.org/2.0/index.html#org.scalatest.concurrent.Futures
Specs2 also has support for testing Futures:
http://etorreborre.github.io/specs2/guide/org.specs2.guide.Matchers.html
ScalaTest 3.x supports asynchronous non-blocking testing.
I have an iteration vals: Iterable[T] and a long-running function without any relevant side effects: f: (T => Unit). Right now this is applied to vals in the obvious way:
vals.foreach(f)
I would like the calls to f to be done concurrently (within reasonable limits). Is there an obvious function somewhere in the Scala base library? Something like:
Concurrent.foreach(8 /* Number of threads. */)(vals, f)
While f is reasonably long running, it is short enough that I don't want the overhead of invoking a thread for each call, so I am looking for something based on a thread pool.
Many of the answers from 2009 still use the old scala.actors.Futures._, which are no longer in the newer Scala. While Akka is the preferred way, a much more readable way is to just use parallel (.par) collections:
vals.foreach { v => f(v) }
becomes
vals.par.foreach { v => f(v) }
Alternatively, using parMap can appear more succinct though with the caveat that you need to remember to import the usual Scalaz*. As usual, there's more than one way to do the same thing in Scala!
Scalaz has parMap. You would use it as follows:
import scalaz.Scalaz._
import scalaz.concurrent.Strategy.Naive
This will equip every functor (including Iterable) with a parMap method, so you can just do:
vals.parMap(f)
You also get parFlatMap, parZipWith, etc.
I like the Futures answer. However, while it will execute concurrently, it will also return asynchronously, which is probably not what you want. The correct approach would be as follows:
import scala.actors.Futures._
vals map { x => future { f(x) } } foreach { _() }
I had some issues using scala.actors.Futures in Scala 2.8 (it was buggy when I checked). Using java libs directly worked for me, though:
final object Parallel {
val cpus=java.lang.Runtime.getRuntime().availableProcessors
import java.util.{Timer,TimerTask}
def afterDelay(ms: Long)(op: =>Unit) = new Timer().schedule(new TimerTask {override def run = op},ms)
def repeat(n: Int,f: Int=>Unit) = {
import java.util.concurrent._
val e=Executors.newCachedThreadPool //newFixedThreadPool(cpus+1)
(0 until n).foreach(i=>e.execute(new Runnable {def run = f(i)}))
e.shutdown
e.awaitTermination(Math.MAX_LONG, TimeUnit.SECONDS)
}
}
I'd use scala.actors.Futures:
vals.foreach(t => scala.actors.Futures.future(f(t)))
The latest release of Functional Java has some higher-order concurrency features that you can use.
import fjs.F._
import fj.control.parallel.Strategy._
import fj.control.parallel.ParModule._
import java.util.concurrent.Executors._
val pool = newCachedThreadPool
val par = parModule(executorStrategy[Unit](pool))
And then...
par.parMap(vals, f)
Remember to shutdown the pool.
You can use the Parallel Collections from the Scala standard library.
They're just like ordinary collections, but their operations run in parallel. You just need to put a par call before you invoke some collections operation.
import scala.collection._
val array = new Array[String](10000)
for (i <- (0 until 10000).par) array(i) = i.toString