use dispatch 0.9.5 behind proxy? - scala

I'm trying to execute (in IntelliJ IDE or from sbt command-line) this very basic dispatch snippet from behind a proxy :
import dispatch._
val svc = url("http://api.hostip.info/country.php")
val country = Http(svc > as.String)
println(country())
and all I can get is an exception :
java.net.ConnectException: Connection timed out: no further information to
http://api.hostip.info/country.php java.util.concurrent.ExecutionException:
java.net.ConnectException: Connection timed out: no further information
to http://api.hostip.info/country.php
I tried with no conclusive result to set the usual vm parameters :
-Dhttp.proxyHost=_my_proxy_host_ -Dhttp.proxyPort=80
and still got the same exception.
On the other hand, the following snippet does work well :
import dispatch._
val svc = url("http://api.hostip.info/country.php") setProxyServer(new com.ning.http.client.ProxyServer(myproxyhost,80))
val country = Http(svc > as.String)
println(country())
Since it does not seem quite aesthetic nor scala-ish, I wonder if it is really what I am supposed to do in such a case.
Any help would be welcome, thanks in advance.

http.proxyHost and http.proxyPort will be used if you set this parameter:
-Dcom.ning.http.client.AsyncHttpClientConfig.useProxyProperties=true
Additionaly there are parameters:
-Dcom.ning.http.client.AsyncHttpClientConfig.proxy.user=user
-Dcom.ning.http.client.AsyncHttpClientConfig.proxy.password=password
-Dcom.ning.http.client.AsyncHttpClientConfig.proxy.protocol=NTLM

Looks like my question wasn't very inspiring.
I did a little exercie in the pimp my library style :
package dispatch
package object ext {
import com.ning.http.client.ProxyServer
class DefaultPoxyVerbs(override val subject: Req) extends ProxyVerbs
object NoProxy extends ProxyServer("",0)
object Proxy {
def apply(host: String,port: Int) = new ProxyServer(host,port)
}
trait ProxyVerbs extends RequestVerbs {
def through(proxy : ProxyServer) =
if (NoProxy == proxy) subject else subject.setProxyServer(proxy)
def ||>(proxy : ProxyServer) = through(proxy)
}
implicit def implyProxyVerbs(builder: Req) =
new DefaultPoxyVerbs(builder)
}
Now I can write :
import dispatch._
import dispatch.ext._
val svc = url("http://api.hostip.info/country.php") through Proxy("blah blah blah",80)
val country = Http(svc > as.String)
println(country())
which is a little bit more eye pleasing and coherent regarding dispatch api style.
While it was an interesting exercise, I still don't know by now if i was originally using the api the way I was supposed to nor why setting http.proxyHost and http.proxyPort properties didn't work since it seems to work for others.

Related

Scala "constructor DCAwareRoundRobinPolicy in class DCAwareRoundRobinPolicy cannot be accessed in object CassandraConnector"

I am a scala newbie and really need help with an issue I've been experiencing with some code. I have set up the following code to create cassandra connections:
package some.package
import java.net.InetAddress
import java.util
import com.datastax.driver.core._
import com.datastax.driver.core.policies.{DCAwareRoundRobinPolicy, TokenAwarePolicy}
import scala.collection.JavaConversions._
object CassandraConnector {
def getCluster(contactPointIpString: String, preferred_dc: String): Cluster = {
val contactPointIpStrings = contactPointIpString.split(",").toList
val contactPointIpList = contactPointIpStrings.flatMap { ipAddress: String => InetAddress.getAllByName(ipAddress) }
println(s"Building Cluster w/ Contact Points: $contactPointIpList")
Cluster.builder()
.addContactPoints(contactPointIpList)
.withClusterName("multi_dc_user_data")
.withLoadBalancingPolicy(new TokenAwarePolicy(new DCAwareRoundRobinPolicy(preferred_dc)))
.withQueryOptions(new QueryOptions().setConsistencyLevel(ConsistencyLevel.LOCAL_ONE))
.build()
}
}
While building the project in IntelliJ, the following error happens:
constructor DCAwareRoundRobinPolicy in class DCAwareRoundRobinPolicy cannot be accessed in object CassandraConnector
.withLoadBalancingPolicy(new TokenAwarePolicy(new DCAwareRoundRobinPolicy(preferred_dc)))
Found something similar to my problem here, and tried to change my code to the following:
val dcAwareRoundRobinPolicyBuilder = new DCAwareRoundRobinPolicy.Builder
val dcAwareRoundRobinPolicy = dcAwareRoundRobinPolicyBuilder.withLocalDc(preferred_dc)
Cluster.builder()
.addContactPoints(contactPointIpList)
.withClusterName("multi_dc_user_data")
.withLoadBalancingPolicy(new TokenAwarePolicy(dcAwareRoundRobinPolicy.build()))
.withQueryOptions(new QueryOptions().setConsistencyLevel(ConsistencyLevel.LOCAL_ONE))
.build()
This resolved the issue and now the build finishes successfully but I am not so sure if what I've done is necessarily correct, so I'd really appreciate any help in this regard.
DCAwareRoundRobinPolicy uses the Builder Pattern to decouple the public API for instance creation from the particular implementation of the constructors for the instances. This is accomplished by not exposing a public constructor and instead only allowing DCAwareRoundRobinPolicy.Builder to construct instances of DCAwareRoundRobinPolicy.
You're using it as it's intended (assuming that you're passing options which match what you're expecting).

How can I resolve conflicting actor systems while testing akka-http and akka actors at the same spec file?

I have a Route defined using akka-http that uses an actor inside to send messages.
My route looks like this:
path("entity") {
post {
entity(as[Enrtity]) {
entity =>
val result: Future[Message] = mainActor.ask {
ref: akka.actor.typed.ActorRef[Message] =>
Message(
entity = entity,
replyRef = ref
)
}
complete("OK")
}
}
}
My test spec:
class APITest
extends ScalaTestWithActorTestKit(ManualTime.config)
with ScalatestRouteTest
with AnyWordSpecLike {
val manualTime: ManualTime = ManualTime()
// my tests here ...
}
Compiling the test fails since there are conflicting actor systems:
class APITest inherits conflicting members:
[error] implicit def system: akka.actor.typed.ActorSystem[Nothing] (defined in class ActorTestKitBase) and
[error] implicit val system: akka.actor.ActorSystem (defined in trait RouteTest)
Overriding the actor system doesn't help either since the inherited actor systems are of both typed and untyped ones.
How can I resolve this easily?
Update:
This is related to conflicting inherited members with different types, but we might be able to solve what I want to achieve in this context differently.
I spent a little time here while moving over to typed. For anyone still looking, there's a nice hint at https://developer.lightbend.com/guides/akka-http-quickstart-scala/testing-routes.html
// the Akka HTTP route testkit does not yet support a typed actor system (https://github.com/akka/akka-http/issues/2036)
// so we have to adapt for now
lazy val testKit = ActorTestKit()
implicit def typedSystem = testKit.system
override def createActorSystem(): akka.actor.ActorSystem =
testKit.system.classicSystem
Looking at the first comment at https://github.com/akka/akka-http/issues/2036 it notes
Perhaps just a docs addition to show that you don't need to use the ActorTestKit from Akka Typed and can just use TestProbes e.g. https://gist.github.com/chbatey/964b80adc2cd124fa4bf4624927b5be0
or val probe = TestProbe[Ping]() > val probe = testKit.createTestProbe[Ping]()

Logging within Akka TestKit outside Actors

I have been trying to Log things within my scalaTest as such:
class ChangeSetActorTest extends PersistenceSpec(ActorSystem("Persistent-test-System")) with PersistenceCleanup {
val log = Logging(system, this)
Basically let's just say that ChangesetActorTest inherit from TestKit(system)
Unfortunately Logging(system, this) does not work with the this.
I get the following error:
[error]
/Users/maatary/Dev/IdeaProjects/PoolpartyConnector/src/test/scala/org/iadb/poolpartyconnector/changepropagation/ChangeSetActorTest.scala:22:
Cannot find LogSource for
org.iadb.poolpartyconnector.changepropagation.ChangeSetActorTest
please see ScalaDoc for LogSource for how to obtain or construct one.
[error] val log = Logging(system, this)
I believe in the Akka Logging Doc this is the following point:
and in all other cases a compile error occurs unless and implicit LogSource[T] is in scope for the type in question.
In other words there is no LogSource[TestKit]
I would like the simplest solution to deal with that issue, with minimal additional configuration. So far what i did is the following and everything works as expected:
class ChangeSetActorTest extends PersistenceSpec(ActorSystem("Persistent-test-System")) with PersistenceCleanup {
val log = system.log
From there I just go and do things like
val received = chgtFetcher.receiveWhile((requestInterval + ProcessingLag).*(3)) {
case msg:FetchNewChangeSet => log.info(s"received: ${msg}" ) ; chgtFetcher.reply(NoAvailableChangeSet); msg
}
My question, is this recommended approach. So far the order of the message coming from my actor and the one from the test are well ordered.
What is the recommended approach to log in a unified way:
From the Test class (e.g. above) and the Actor at the same time ?
If one uses a system where external class needs to log as well and we need one unified logging (asynchronous) going on.
Have a look at this comment:
https://github.com/akka/akka/blob/master/akka-actor/src/main/scala/akka/event/Logging.scala#L196-L237
I believe a more straight forward approach would be to define your implicit LogSource[ChangeSetActorTest] locally.
I.E:
val log = {
implicit val logSource = new LogSource[ChangeSetActorTest] {
override def genString(t: ChangeSetActorTest) = "ChangeSetActorTest"
}
Logging(system, this)
}
Simplest way to log in a TestKit is either:
Get the logger from underlyingActor:
val mockActor = TestActorRef(new XXXActor)
val log = mockActor.underlyingActor.log
Use FeatureSpecLike
http://doc.scalatest.org/3.0.1-2.12/org/scalatest/FeatureSpecLike.html
class ChangeSetActorTest extends PersistenceSpec(ActorSystem("Persistent-test-System")) with PersistenceCleanup with FeatureSpecLike {
//...
alert("Something like warning")
info("Infos")
note("Green infos")
markup("documents")
}

https4s how to make a POST call to a REST web service

I am trying to use the http4s library. I am trying to make a POST request to a REST web service with some json payload.
when I read the documentation http://http4s.org/docs/0.15/ I can only see a GET method example.
does anyone know how to make a POST?
It looks like the get/getAs methods mentioned in the example are just convenience wrappers for the fetch method. See https://github.com/http4s/http4s/blob/a4b52b042338ab35d89d260e0bcb39ccec1f1947/client/src/main/scala/org/http4s/client/Client.scala#L116
Use the Request constructor and pass Method.POST as the method.
fetch(Request(Method.POST, uri))
https4s version: 0.14.11
The hard part is how to set the post body. When you dive into the code, you may find type EntityBody = Process[Task, ByteVector]. But wtf is it? However, if you have not been ready to dive into scalaz, just use withBody.
object Client extends App {
val client = PooledHttp1Client()
val httpize = Uri.uri("http://httpize.herokuapp.com")
def post() = {
val req = Request(method = Method.POST, uri = httpize / "post").withBody("hello")
val task = client.expect[String](req)
val x = task.unsafePerformSync
println(x)
}
post()
client.shutdownNow()
}
P.S. my helpful post about http4s client(Just skip the Chinese and read the scala code): http://sadhen.com/blog/2016/11/27/http4s-client-intro.html
import org.http4s.circe._
import org.http4s.dsl._
import io.circe.generic.auto._
case class Name(name: String)
implicit val nameDecoder: EntityDecoder[Name] = jsonOf[Name]
def routes: PartialFunction[Request, Task[Response]] = {
case req # POST -> Root / "hello" =>
req.decode[Name] { name =>
Ok(s"Hello, ${name.name}")
}
Hope this helps.

Akka Http Route Test: Request was neither completed nor rejected within 1 second

I am trying to write a test case for my application with akka-http. One of the testcases is given below:
import akka.http.scaladsl.model.headers.RawHeader
import akka.http.scaladsl.testkit.{ ScalatestRouteTest}
import com.reactore.common.core.{CommonCoreSystem, CommonActors, BootedCommonCoreSystem}
import scala.concurrent.duration._
import com.reactore.common.feature.referencedata.{DepartmentRepository, DepartmentService, DepartmentController, DepartmentRest}
import org.scalatest.concurrent.AsyncAssertions
import org.scalatest.time.Span
import org.scalatest.{WordSpec, Matchers}
import akka.http.scaladsl.model.StatusCodes._
/**
* Created by krishna on 18/6/15.
*/
class DepartmentITTest extends WordSpec with Matchers with ScalatestRouteTest with CommonCoreSystem with CommonActors {
// override val departmentRouter = system.actorOf(Props(classOf[DepartmentService], DepartmentRepository), Constants.DEPARTMENT_ROUTER_NAME)
val deptRoute = (new DepartmentRest(departmentRouter)(DepartmentController).deptRoutes)
implicit val timeout = AsyncAssertions.timeout(Span.convertDurationToSpan(5.second))
val departmentJson = """{"id":13,"name":"ENGP22","shortCode":"ENGP2","parentDepartmentId":null,"mineId":null,"photoName":null,"isRemoved":false,"isPendingForApproval":false,"createdBy":0,"createDate":"2015-03-09 00:00:00","modifiedBy":0,"modifiedDate":"2015-03-09 00:00:00"}"""
val header = RawHeader("apiKey", "xxxxx")
"Service" should {
"return department by id" in {
Get("/departments/13").withHeaders(header) ~> deptRoute ~> check {
// Thread.sleep(500)
status shouldBe OK
responseAs[String] shouldBe departmentJson
}
}
}
}
When I run this, it works correctly sometimes, and sometimes I get the error as Request was neither completed nor rejected within 1 second. I added a Thread.sleep for making it work now. I know that is not the correct solution. Could anyone tell me how to make the test wait for more than 1 second?
The following is working for me:
import akka.actor.ActorSystem
import scala.concurrent.duration._
import spray.testkit.ScalatestRouteTest
class RouteSpec extends ScalatestRouteTest {
implicit def default(implicit system: ActorSystem) = RouteTestTimeout(2.second)
...
You can use the "eventually" matcher in ScalaTest to wait for a condition to become true:
eventually { status shouldBe OK }
http://www.artima.com/docs-scalatest-2.0.M5/org/scalatest/concurrent/Eventually.html
That should suffice if the Thread.sleep you commented out above fixes things for you.
However, it looks to me like the actual error is that the timeout that the RouteTest trait is using is too short. The error message "Request was neither completed nor rejected within 1 second." comes from RouteTestResultComponent via akka.http.scaladsl.testkit.RouteTest.
I think that the Thread.sleep is a distraction. The default timeout for routing tests is 1 second; see akka.http.scaladsl.testkit.RouteTestTimeout.default. You provide a 5 second implicit timeout in your code, but I think it's of a different type. Try making a RouteTestTimeout implicitly available with a longer timeout.
you can simply update the config to dilate the timeout
akka {
test {
# factor by which to scale timeouts during tests, e.g. to account for shared
# build system load
timefactor = 3.0
}
}