Why can't the compiler find implicit ExecutionContext with Implicits.global imported? - scala

I have the following piece of code:
import java.util.concurrent.Executor
import scala.concurrent.Future
trait Storage {
def store(location: String, content: Array[Byte])(implicit exec: Executor): Future[String]
}
object LocalStorage extends Storage {
override def store(location: String, content: Array[Byte])(implicit exec: Executor): Future[String] =
Future {
... do some stuff ...
"Hello"
}
}
And here comes the code for testing:
object LocalStorageTest extends App{
import scala.concurrent.ExecutionContext.Implicits.global
val testImage = AsyncWebClient get "some url"
val result = testImage.flatMap{content => LocalStorage.store("logo.png",content)}
val status =Await.result(result, 30 seconds)
println(status)
AsyncWebClient.shutDown
}
Whenever I try to run the code, I am getting the following error:
Cannot find an implicit ExecutionContext.
Why? Isn't scala.concurrent.ExecutionContext.Implicits.global in scope already? When this import is added directly to LocalStorage object, it works (!)
BUT I am loosing a possibility to alter the context. Since I do not want to run this code on global every time. As this is a part of an akka application and for "production" runtime it is supposed to run on some dispatcher. Just for testing I would like to run it on a global execution context.
Am I missing some important concept here or is my design wrong that I am loosing such flexibility? Where is the issue?!

An ExecutionContext is not the same thing as an Executor. You should make the method accept an implicit exec: ExecutionContext (scala.concurrent.ExecutionContext) and then it will behave as you expect.

Related

Scala: mock Hadoop File System

I need to mock (with scalamock 4.4.0/scalatest 3.2.9) the behaviour of Hadoop file system. I found out, that I can't mock it directly, so I'm trying to get around this problem.
On the first, try I created wrapper (in the scope of main logic):
class HadoopFsWrapper(path: Path)(implicit fs: org.apache.hadoop.fs.FileSystem) {
def listStatus(path) = fs.listStatus(path)
}
class HadoopService(path: Path)(implicit val fs: HadoopFsWrapper){
def someLogic() = fs.listStatus(path)
}
I declare this class both in main and test code with defining of implicit HadoopFsWrapper.
implicit val fs = mock[HadoopFsWrapper]
val hs = HadoopService(path)
// other test logic
And this done well.
But I don't think, its a good idea to modify main code for test purpose.
Other way I considered is to create wrapper only in test scope and apply some kind of "decorator" to deceive HadoopService, which expects FileSystem to be defined in main scope:
// only in test scope
class HadoopFsWrapper(path: Path) extends org.apache.hadoop.fs.FileSystem {
def listStatus(path) = listStatus(path)
// dummies for implemented FileSystem-methods
}
object HadoopFsWrapper {
def apply(): org.apache.hadoop.fs.FileSystem = new HadoopFsWrapper
}
// main logic
class HadoopService(path: Path)(implicit val fs: org.apache.hadoop.fs.FileSystem){
def someLogic() = fs.listStatus(path)
}
But when I'm trying to mock HadoopFsWrapper it's defined as null.

Phantom dsl 2.24 tables are not created

Recently trying to migrate to the latest phantom version 2.24.8. I created a dummy project, but running into a few issues that I can't figure out. Here's my code:
import com.outworkers.phantom.connectors.{CassandraConnection, ContactPoints}
import com.outworkers.phantom.database.Database
import scala.concurrent.Future
import com.outworkers.phantom.dsl._
case class Test(id: String, timestamp: String)
abstract class Tests extends Table[Tests, Test] {
object id extends StringColumn with PartitionKey
object timestamp extends StringColumn with ClusteringOrder
}
abstract class ConcreteTests extends Tests with RootConnector {
def addTest(l: Test): Future[ResultSet] = {
// store(l).consistencyLevel_=(ConsistencyLevel.LOCAL_ONE).future
insert.value(_.id, l.id)
.value(_.timestamp, l.timestamp)
.consistencyLevel_=(ConsistencyLevel.QUORUM).future
}
}
class MyDB(override val connector: CassandraConnection) extends Database[MyDB](connector) {
object tests extends ConcreteTests with connector.Connector
def init(): Unit = {
tests.create
}
}
object Test{
def main(args: Array[String]): Unit = {
val db = new MyDB(ContactPoints(Seq("127.0.0.1")).keySpace("tests"))
db.init
db.tests.addTest(Test("1", "1323234234"))
println("Done")
}
}
It compiled and ran in IntelliJ and print out 'Done'. However, no table is ever created. Also no exceptions or warnings. It did nothing. I tried to stop the local cassandra database. The code throws the NoHostAvailableException. So it does try to connect the local database. What is the problem?
Another weird thing is that "com.typesafe.play" %% "play-json" % "2.6.9" is in my build.sbt. If I remove the library, the same code throws the following exception:
Exception in thread "main" java.lang.NoClassDefFoundError: scala/reflect/runtime/package$
at com.outworkers.phantom.column.AbstractColumn.com$outworkers$phantom$column$AbstractColumn$$_name(AbstractColumn.scala:55)
at com.outworkers.phantom.column.AbstractColumn.com$outworkers$phantom$column$AbstractColumn$$_name$(AbstractColumn.scala:54)
at com.outworkers.phantom.column.Column.com$outworkers$phantom$column$AbstractColumn$$_name$lzycompute(Column.scala:22)
at com.outworkers.phantom.column.Column.com$outworkers$phantom$column$AbstractColumn$$_name(Column.scala:22)
at com.outworkers.phantom.column.AbstractColumn.name(AbstractColumn.scala:58)
at com.outworkers.phantom.column.AbstractColumn.name$(AbstractColumn.scala:58)
at com.outworkers.phantom.column.Column.name(Column.scala:22)
at com.outworkers.phantom.builder.query.InsertQuery.value(InsertQuery.scala:107)
Really cannot figure what's going on. Any help?
BTW, I'm using scala 2.12.6 and JVM 1.8.181.
You're not using the correct DSL method for table creation, have a look at the official guide. All that table.create does is to create an empty CreateQuery, and you're coercing the return type to Unit manually.
The automated blocking create method is on Database, not on table, so what you want is:
class MyDB(override val connector: CassandraConnection) extends Database[MyDB](connector) {
object tests extends ConcreteTests with connector.Connector
def init(): Unit = {
this.create
}
}
If you want to achieve the same thing using table, you need:
class MyDB(override val connector: CassandraConnection) extends Database[MyDB](connector) {
object tests extends ConcreteTests with connector.Connector
def init(): Unit = {
import scala.concurrent.duration._
import scala.concurrent.Await
Await.result(tests.create.future(), 10.seconds)
}
}
It's only the call to future() method that will trigger any kind of action to the database, otherwise you're just building a query without executing it. The method name can be confusing, and we will improve the docs and future releases to make it more obvious.
The conflict with play 2.6.9 looks very weird, it's entirely possible there's an incompatible dependency behind the scenes to do with macro compilation. Raise that as a separate issue and we can definitely have a look at it.

scalatest Flatspec: Timeout for entire class

With the scalatest Flatspec and the TimeLimits trait I can set a timeout for a line of code like so:
import org.scalatest.time.SpanSugar._
import org.scalatest.concurrent.TimeLimits
import org.scalatest.FlatSpec
class MyTestClass extends Flatspec with TimeLimits {
"My thread" must "complete on time" in {
failAfter(100 millis) { infiniteLoop() }
// I have also tried cancelAfter
}
}
This should fail due to a timeout. However, when I run this test in Intellij it just runs forever.
Furthermore, I do not want to have to rewrite the timeout for every method, instead I would like to configure it once for the entire class. The PatienceConfig claims to do that, but it does not seem to do anything. The test is still runs forever.
import org.scalatest.FlatSpec
import org.scalatest.time.{Millis, Span}
import org.scalatest.concurrent.{Eventually, IntegrationPatience}
class MyTestClass extends Flatspec with Eventually with IntegrationPatience {
implicit val defaultPatience = PatienceConfig(timeout=Span(100, Millis))
"My thread" must "complete on time" in {
inifiniteLoop()
}
}
I looked for a solution myself. came a cross this answer, it worked for me.
I am using flatspec, added the trait TimeLimitedTests
with TimeLimitedTests
then inside the code I wrote my limit for each of the tests val timeLimit: Span = Span(2000, Millis)
which is defined by the trait (we are overriding it).
Finally it didn't work until I overriden the interrupter as suggested by Rumoku in the referenced answer by
override val defaultTestInterruptor: Interruptor = new Interruptor {
override def apply(testThread: Thread): Unit = {
println("Kindly die")
testThread.stop() // deprecated. unsafe. do not use
}}
I hope this helps

Why are implicit variables not initialized in Scala when called from unit test?

Given the the following singleton Object in Scala:
package demo
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.server.Directives._
import akka.stream.ActorMaterializer
import scala.concurrent.Future
import scala.io.StdIn
object WebServer extends App {
implicit val system = ActorSystem("myActorSystem")
implicit val executionContext = system.dispatcher
implicit val materializer = ActorMaterializer()
val route = {
path("api" / "done-as-promised") {
get {
complete {
Future.successful("done")
}
}
}
}
val bindingFuture = Http().bindAndHandle(route, "localhost", 8080)
}
And the following unit test
package demo
import akka.http.scaladsl.testkit.ScalatestRouteTest
import org.scalactic.TypeCheckedTripleEquals
import org.scalatest.{Inspectors, Matchers, WordSpec}
class WebServerSpec extends WordSpec with Matchers with TypeCheckedTripleEquals with Inspectors with ScalatestRouteTest {
"The WebServer /done-as-promised" should {
"return done" in {
// tests:
Get("/api/done-as-promised") ~> WebServer.route ~> check {
status.intValue() shouldEqual 200
responseAs[String] shouldEqual "done"
}
}
}
}
I get the following error:
[ERROR] [04/19/2016 07:12:18.995]
[ScalaTest-run-running-WebServerSpec]
[akka.actor.ActorSystemImpl(demo-WebServerSpec)] Error during
processing of request
HttpRequest(HttpMethod(GET),http://example.com/api/done-as-promised,List(),HttpEntity.Strict(none/none,ByteString()),HttpProtocol(HTTP/1.1))
java.lang.NullPointerException at
akka.http.scaladsl.server.directives.ExecutionDirectives$$anonfun$handleExceptions$1$$anonfun$apply$1.apply(ExecutionDirectives.scala:33)
at
akka.http.scaladsl.server.directives.ExecutionDirectives$$anonfun$handleExceptions$1$$anonfun$apply$1.apply(ExecutionDirectives.scala:29)
at
akka.http.scaladsl.testkit.RouteTest$TildeArrow$$anon$1.apply(RouteTest.scala:162)
at
akka.http.scaladsl.testkit.RouteTest$TildeArrow$$anon$1.apply(RouteTest.scala:150)
It took me a while to figure out. The thing is: removing the extends app will make the test succeed.
The reason for the problem is that when WebServer is declared as extends App, it uses the DelayedInit functionality of the App trait. Because of this, the initialization code in the contructor is not added to the constructor of the WebServer object. Instead is called when the main method is called on the WebServer. So when he references the "route" inside the tests, those are all coming up null.
Mixing in the DelayedInit trait (App extends from DelayedInit) will rewrite your class or object template. Instead of adding your val's and var's to the constructor, it will be added to the def delayedInit(body: => Unit) hook (inaccessible to user code). Apparently this one is called whenever the main method is called.
You can verify this by simply calling "main" on the WebServer inside the test. If you do this, then the test will pass. This is because calling main triggers the initialization resulting in those objects being created.
Generally speaking though the right solution is probably to move the routing to somewhere else, rather than having it inside of the base App.

Akka Http Route Test: Request was neither completed nor rejected within 1 second

I am trying to write a test case for my application with akka-http. One of the testcases is given below:
import akka.http.scaladsl.model.headers.RawHeader
import akka.http.scaladsl.testkit.{ ScalatestRouteTest}
import com.reactore.common.core.{CommonCoreSystem, CommonActors, BootedCommonCoreSystem}
import scala.concurrent.duration._
import com.reactore.common.feature.referencedata.{DepartmentRepository, DepartmentService, DepartmentController, DepartmentRest}
import org.scalatest.concurrent.AsyncAssertions
import org.scalatest.time.Span
import org.scalatest.{WordSpec, Matchers}
import akka.http.scaladsl.model.StatusCodes._
/**
* Created by krishna on 18/6/15.
*/
class DepartmentITTest extends WordSpec with Matchers with ScalatestRouteTest with CommonCoreSystem with CommonActors {
// override val departmentRouter = system.actorOf(Props(classOf[DepartmentService], DepartmentRepository), Constants.DEPARTMENT_ROUTER_NAME)
val deptRoute = (new DepartmentRest(departmentRouter)(DepartmentController).deptRoutes)
implicit val timeout = AsyncAssertions.timeout(Span.convertDurationToSpan(5.second))
val departmentJson = """{"id":13,"name":"ENGP22","shortCode":"ENGP2","parentDepartmentId":null,"mineId":null,"photoName":null,"isRemoved":false,"isPendingForApproval":false,"createdBy":0,"createDate":"2015-03-09 00:00:00","modifiedBy":0,"modifiedDate":"2015-03-09 00:00:00"}"""
val header = RawHeader("apiKey", "xxxxx")
"Service" should {
"return department by id" in {
Get("/departments/13").withHeaders(header) ~> deptRoute ~> check {
// Thread.sleep(500)
status shouldBe OK
responseAs[String] shouldBe departmentJson
}
}
}
}
When I run this, it works correctly sometimes, and sometimes I get the error as Request was neither completed nor rejected within 1 second. I added a Thread.sleep for making it work now. I know that is not the correct solution. Could anyone tell me how to make the test wait for more than 1 second?
The following is working for me:
import akka.actor.ActorSystem
import scala.concurrent.duration._
import spray.testkit.ScalatestRouteTest
class RouteSpec extends ScalatestRouteTest {
implicit def default(implicit system: ActorSystem) = RouteTestTimeout(2.second)
...
You can use the "eventually" matcher in ScalaTest to wait for a condition to become true:
eventually { status shouldBe OK }
http://www.artima.com/docs-scalatest-2.0.M5/org/scalatest/concurrent/Eventually.html
That should suffice if the Thread.sleep you commented out above fixes things for you.
However, it looks to me like the actual error is that the timeout that the RouteTest trait is using is too short. The error message "Request was neither completed nor rejected within 1 second." comes from RouteTestResultComponent via akka.http.scaladsl.testkit.RouteTest.
I think that the Thread.sleep is a distraction. The default timeout for routing tests is 1 second; see akka.http.scaladsl.testkit.RouteTestTimeout.default. You provide a 5 second implicit timeout in your code, but I think it's of a different type. Try making a RouteTestTimeout implicitly available with a longer timeout.
you can simply update the config to dilate the timeout
akka {
test {
# factor by which to scale timeouts during tests, e.g. to account for shared
# build system load
timefactor = 3.0
}
}