How do I send a POST request to twitter API using Play Framework 2.0 (with Scala)? The API I'm trying to call work with both GET and POST, and I've successfully called it using GET with this code:
val followersURL = "http://api.twitter.com/1/users/lookup.json?user_id=" + listOfFollowers.mkString(",")
WS.url(followersURL)
.sign(OAuthCalculator(Twitter.KEY, tokens))
.get()
.map{ response =>
val screenName: Seq[String] = response.json match {
case res: JsArray => res.value.map{ value => (value \ "name").toString }
case _ => Seq("")
}
}
Then I tried to call the API using POST like this:
WS.url("http://api.twitter.com/1/users/lookup.json")
.sign(OAuthCalculator(Twitter.KEY, tokens))
.post(Map("user_id"->listOfFollowers))
.map { response =>
val screenName: Seq[String] = response.json match {
case res: JsArray => res.value.map{ value => (value \ "name").toString }
case _ => Seq("")
}
}
It didn't work and I get this exception:
[error] play - Waiting for a promise, but got an error: null
java.lang.NullPointerException: null
at java.io.Reader.<init>(Unknown Source) ~[na:1.7.0_01]
at java.io.InputStreamReader.<init>(Unknown Source) ~[na:1.7.0_01]
at oauth.signpost.OAuth.decodeForm(OAuth.java:157) ~[signpost-core.jar:na]
at oauth.signpost.AbstractOAuthConsumer.collectBodyParameters(AbstractOAuthConsumer.java:236) ~[signpost-core.jar:na]
at oauth.signpost.AbstractOAuthConsumer.sign(AbstractOAuthConsumer.java:96) ~[signpost-core.jar:na]
at play.api.libs.oauth.OAuthCalculator.sign(OAuth.scala:106) ~[play_2.9.1.jar:2.0.1]
Since it says that the exception occurs on the OAuthCalculator, I try to comment out the .sign call, and it didn't throw any exception, but of course I didn't get the right result.
Am I doing something wrong? What am I doing wrong, and why? How could I fix the problem?
Thanks before.
I have found this to work:
WS.url("http://api.twitter.com/1/users/lookup.json?user_id="+listOfFollowers)
.sign(OAuthCalculator(Twitter.KEY, tokens))
.post("ignored")
.map { response =>
val screenName: Seq[String] = response.json match {
case res: JsArray => res.value.map{ value => (value \ "name").toString }
case _ => Seq("")
}
}
I've also made notes to revisit my code with every major upgrade of Play! to check if the above gets fixed, because this is obviously not right.
Once you use the Play Framework tools for getting your user's token and secret, you can then use the twitter4j ("org.twitter4j" % "twitter4j-core" % "3.0.3") library to do your posting like this.
import twitter4j.conf.ConfigurationBuilder
import twitter4j.{StatusUpdate, TwitterFactory}
val config = new ConfigurationBuilder()
.setOAuthConsumerKey(twitterKey)
.setOAuthConsumerSecret(twitterSecret)
.setOAuthAccessToken(token)
.setOAuthAccessTokenSecret(secret)
.build()
val twitter = new TwitterFactory(config).getInstance()
val status = new StatusUpdate(tweet)
status.media(photoName, stream)
val twitResp = twitter.updateStatus(status)
Annoying to have to use two libraries, and twitter4j isn't async so it's slightly less resource-efficient, but it lets you do real posts which are necessary sometimes.
Related
I am using Akka HTTP cache to cache my result. But i am facing issue to test it.
class GoogleAnalyticsController #Inject()(cache: Cache[String, HttpResponse],
googleAnalyticsApi: GoogleAnalyticsTrait,
googleAnalyticsHelper: GoogleAnalyticsHelper)
(implicit system: ActorSystem, materializer: ActorMaterializer) {
def routes: Route =
post {
pathPrefix("pageviews") {
path("clients" / Segment) { accountsClientId =>
entity(as[GoogleAnalyticsMetricsRequest]) { googleAnalyticsMetricsRequest =>
val googleAnalyticsMetricsKey = "key"
complete(
cache.getOrLoad(googleAnalyticsMetricsKey, _ => getGoogleAnalyticsMetricsData(accountsClientId, googleAnalyticsMetricsRequest))
)
}
}
}
}
private def getGoogleAnalyticsMetricsData(accountsClientId: String,
request: GoogleAnalyticsMetricsRequest) = {
val payload = generate(request)
val response = googleAnalyticsApi.googleAnalyticsMetricResponseHandler(accountsClientId, payload) // response from another microservice
googleAnalyticsHelper.googleAnalyticsMetricResponseHandler(
googleAnalyticsMetricsRequest.metricName, response)
}
}
class GoogleAnalyticsHelper extends LoggingHelper {
def googleAnalyticsMetricResponseHandler(metricName: String, response: Either[Throwable, Long]): Future[HttpResponse] =
response.fold({ error =>
logger.error(s"An exception has occurred while getting $metricName from behavior service and error is ${error.getMessage}")
Marshal(FailureResponse(error.getMessage)).to[HttpResponse].map(httpResponse => httpResponse.copy(status = StatusCodes.InternalServerError))
}, value =>
Marshal(MetricResponse(metricName, value)).to[HttpResponse].map(httpResponse => httpResponse.copy(status = StatusCodes.OK))
)
}
Test case: Sharing only the relevant part
"get success metric response for " + pageviews + " metric of given accounts client id" in { fixture =>
import fixture._
val metricResponse = MetricResponse(pageviews, 1)
val eventualHttpResponse = Marshal(metricResponse).to[HttpResponse].map(httpResponse => httpResponse.copy(status = StatusCodes.OK))
when(cache.getOrLoad(anyString, any[String => Future[HttpResponse]].apply)).thenReturn(eventualHttpResponse)
when(googleAnalyticsApi.getDataFromGoogleAnalytics(accountsClientId, generate(GoogleAnalyticsRequest(startDate, endDate, pageviews))))
.thenReturn(ApiResult[Long](Some("1"), None))
when(googleAnalyticsHelper.googleAnalyticsMetricResponseHandler(pageviews, Right(1))).thenReturn(eventualHttpResponse)
Post(s"/pageviews/clients/$accountsClientId").withEntity(requestEntity) ~>
googleAnalyticsController.routes ~> check {
status shouldEqual StatusCodes.OK
responseAs[String] shouldEqual generate(metricResponse)
}
}
By doing this, I am best to test if the cache has the key but not able to test if cache misses the hit. In code coverage, it misses following highlighted part
cache.getOrLoad(googleAnalyticsMetricsKey, _ =>
getGoogleAnalyticsMetricsData(accountsClientId,
googleAnalyticsMetricsRequest))
If there is a design issue, please feel free to guide me on how can I make my design testable.
Thanks in advance.
I think you don't need to mock the cache. You should create an actual object for cache instead of mocked one.
What you have done is, you have mocked the cache, in this case, the highlighted part will be not called as you are providing the mocked value. In the following stubbing, whenever cache.getOrLoad is found, eventualHttpResponse is returned:
when(cache.getOrLoad(anyString, any[String => Future[HttpResponse]].apply)).thenReturn(eventualHttpResponse)
and hence the function getGoogleAnalyticsMetricsData(accountsClientId, googleAnalyticsMetricsRequest) is never called.
I am using the below code to fetch news from a remote site in my Play Scala application server (2.5.x). I am using the suggested mechanism from "https://www.playframework.com/documentation/2.5.x/ScalaWS".
The use case is in 2 steps:
Call remote API-2 to fetch the news content using a cached token.
If the token is invalid/expired, call remote API-1 to get a token and cache it.
Problem: I am not able to avoid step #2 even if the cached token is valid. Therefore I am fetching the token for each call of this Action in listRemoteNews.
If I use if-else conditions, then I get compile errors as below. Note that I have tried for comprehensions, too, but I need to still use if conditions there. Then I get the same compile errors.
[error] /sandbox/git/play-scala-app.repo/app/controllers/remoteapi/ListRemoteNewsController.scala:26: overloaded method value async with alternatives:
[error] [A](bodyParser: play.api.mvc.BodyParser[A])(block: play.api.mvc.Request[A] => scala.concurrent.Future[play.api.mvc.Result])play.api.mvc.Action[A] <and>
[error] (block: play.api.mvc.Request[play.api.mvc.AnyContent] => scala.concurrent.Future[play.api.mvc.Result])play.api.mvc.Action[play.api.mvc.AnyContent] <and>
[error] (block: => scala.concurrent.Future[play.api.mvc.Result])play.api.mvc.Action[play.api.mvc.AnyContent]
[error] cannot be applied to (scala.concurrent.Future[Any])
[error] def listRemoteNews = Action.async {
Code section below (requestOne and requestTwo are methods forming WSRequest and returning Future[WSResponse]):
#Singleton
class ListRemoteNewsController #Inject()(ws: WSClient)(implicit context: ExecutionContext) extends Controller {
def listRemoteNews = Action.async {
println("<--- Stored Token 1: " + remoteToken)
val responseTwo: Future[WSResponse] = requestTwo.withQueryString("token" -> remoteToken).get()
val resp: Future[JsValue] = responseTwo.map {
response =>
response.json
}
resp.map {
response => {
Ok(response)
}
}
println("<-- Get token...")
val responseOne: Future[WSResponse] = requestOne
val resp2: Future[String] = responseOne.map {
response => {
remoteToken = (response.json \ "token_list" \ "token").as[String]
println("<--- Obtained token 2: " + remoteToken)
remoteToken
}
}
val responseThree: Future[WSResponse] = requestTwo.withQueryString("token" -> remoteToken).get()
responseThree.map {
response => {
println("<--- status: " + response.status)
Ok(response.json)
}
}
}
Futures are things that must be composed. Right now, you're making a number of different requests, which each return futures, but you're not returning those futures or combining them together in any way. Mapping a future is pointless if you don't do anything with the new future returned by the map function.
What you need to do is compose your futures, and this is done using methods like flatMap.
So here's roughly what I'm guessing your code should look like:
requestTwo.withQueryString("token" -> remoteToken).get().flatMap { response =>
if (tokenWasValid(response)) {
Future.successful(Ok(response.json))
} else {
requestOne.get().flatMap { tokenResponse =>
val remoteToken = (response.json \ "token_list" \ "token").as[String]
requestTwo.withQueryString("token" -> remoteToken).get()
}.map { response =>
Ok(response.json)
}
}
}
If the above makes no sense whatsoever, then I recommend you stop and read up on Scala futures before doing anything else.
Here's a selection of tutorials I found by doing a quick Google search:
http://code.hootsuite.com/introduction-to-futures-in-scala/
https://danielasfregola.com/2015/04/01/how-to-compose-scala-futures/
https://doc.akka.io/docs/akka/current/scala/futures.html
In Scala I have a call to service in controller which is returning me Future[WSResponse]. I want to make sure service is returning valid result so send Ok(..) otherwise send BadRequest(...). I don't think I can use map. Any other suggestion?
def someWork = Action.async(parse.xml) { request =>
val result:Future[WSResponse] = someService.processData(request.body.toString())
//Need to send back Ok or BadRequest Message
}
EDIT
Solution from #alextsc is working fine. Now moving to test my existing test is failing. It is getting 400 instead of 200.
test("should post something") {
val requestBody = <value>{UUID.randomUUID}</value>
val mockResponse = mock[WSResponse]
val expectedResponse: Future[WSResponse] = Future.successful(mockResponse)
val request = FakeRequest(Helpers.POST, "/posthere").withXmlBody(requestBody)
when(mockResponse.body).thenReturn("SOME_RESPONSE")
when(someService.processData(any[String])).thenReturn(expectedResponse)
val response: Future[Result] = call(controller.someWork , request)
whenReady(response) { response =>
assert(response.header.status == 200)
}
}
You're on the right track and yes, you can use map.
Since you're using Action.async already and your service returns a future as it stands all you need to do is map that future to a Future[Result] so Play can handle it:
def someWork = Action.async(parse.xml) { request =>
someService.processData(request.body.toString()).map {
// Assuming status 200 (OK) is a valid result for you.
case resp : WSResponse if resp.getStatus == 200 => Ok(...)
case _ => BadRequest(...)
}
}
(I note that your service returns WSResponse (from the play ws java library) and not play.api.libs.ws.Response (the scala version of it), hence getStatus and not just status)
I am trying to make a basic http client posting some data to a REST API with Akka HTTP but I cannot make the following code work :
def sendData(e : GenericEvent): Future[Either[(String,StatusCode),(GenericEvent,StatusCode)]] = {
val request = Marshal(e).to[RequestEntity]
val responseFuture: Future[HttpResponse] = request map { req =>
Source.single(HttpRequest(method = HttpMethods.POST, uri = s"/data-ingest", headers = List(auth), entity = req))
.via(dataIngestFlow)
.runWith(Sink.head)
}
responseFuture.flatMap { response =>
response.status match {
case OK => Unmarshal(response.entity).to[GenericEvent].map(Right(_, response.status))
case BadRequest => Future.successful(Left(s"$e.data: incorrect data", response.status))
case _ => Unmarshal(response.entity).to[String].flatMap { entity =>
val error = s"generic event ingest failed with status code ${response.status} and entity $entity"
logger.error(error)
Future.failed(new IOException(error))
}
}
}
I got the following error
polymorphic expression cannot be instantiated to expected type;
[error] found :
[T]akka.stream.scaladsl.Sink[T,scala.concurrent.Future[T]]
[error] required:
akka.stream.Graph[akka.stream.SinkShape[akka.http.scaladsl.model.HttpResponse],akka.http.scaladsl.model.HttpResponse]
[error] .runWith(Sink.head)
Here is the code for the dataIngestFlow
val dataIngestFlow = Http().outgoingConnection(config.endpointUrl,config.endpointPort)
Here is the code on server side :
val routes = {
logRequestResult("akka-http-microservice") {
path("event-ingest") {
post {
entity(as[GenericEvent]) { eventIngest =>
log.info(s"Ingesting {} and publishing event to Kafka topic {}.", eventIngest.eventType,config.kafkaTopic)
kafka ! eventIngest
complete {
eventIngest
}
}~
entity(as[List[GenericEvent]]) { eventIngestList =>
eventIngestList.foreach{ eventIngest=>
log.info(s"Ingesting {} and publishing event List to Kafka topic {}.", eventIngest.eventType,config.kafkaTopic)
kafka ! eventIngest
}
complete {
eventIngestList
}
}
}
}
}
}
I tried another simple client, it builds well but the ingestion stop after 160 events, the server doesn't receive anymore events.
The first problem I see with your example is that RequestEntity does not have a map function. Therefore, the following line
val responseFuture: Future[HttpResponse] = request map { ...
should not compile.
Further, if request is actually a Future (which I infer from the map) then responseFuture is actually of type Future[Future[HttpResponse]] because the stream materializes into its own Future. To solve this problem you can use Future.flatMap instead of map. Namely:
val responseFuture: Future[HttpResponse] = request flatMap { req =>
This is the monadic bind operation within Futures.
To feed Play response I want to pass Enumerator to result's feed method. I need to pass a state from produce/consume iteration step to the next step (or to keep state). Here
http://engineering.klout.com/2013/01/iteratees-in-big-data-at-klout/
I have found an example but am not sure it is thread-safe:
def pagingEnumerator(url:String):Enumerator[JsValue]={
var maybeNextUrl = Some(url) //Next url to fetch
Enumerator.fromCallback[JsValue] ( retriever = {
val maybeResponsePromise =
maybeNextUrl map { nextUrl=>
WS.url(nextUrl).get.map { reponse =>
val json = response.json
maybeNextUrl = (json \ "next_url").asOpt[String]
val code = response.status //Potential error handling here
json
}
}
maybeResponsePromise match {
case Some(responsePromise) => responsePromise map Some.apply
case None => PlayPromise pure None
}
})
}
What is you way to add a state to Play Enumerator? Is this example thread-safe?
(in the above example old Play API related to Promise/Future is used; let's neglect this fact as far as it doesn't influence the issue itself)
You should use Enumerator.unfoldM:
Enumerator.unfoldM(Some(firstURL)) { maybeNextUrl =>
maybeNextUrl map { nextUrl =>
WS.url(nextUrl).get.map { response =>
val json = response.json
val newUrl = (json \ "next_url").asOpt[String]
val code = response.status //Potential error handling here
Some((newUrl, json))
}
} getOrElse (Future.successful(None))
}