I am writing a small http4s client
val client = SimpleHttp1Client()
val uri = Uri.fromString(requestUrl).valueOr(throw _)
val task = POST(uri, UrlForm("username" -> userName, "password" -> password)).map{request => println("request: " + request.body)}
try {
val response = client.expect[String](task).unsafePerformSync
println("token: " + response)
response
} catch {
case e: Exception => println(e.getMessage);"BadToken"
}
The output is like
[info] Running com.researchnow.nova.shield.NovaShieldSetup
[info] Emit(Vector(ByteVector(44 bytes, 0x757365726e616d653d616268737269766173746176612670617373776f72643d41726)))
[info] Failed: unexpected HTTP status: 400 Bad Request
[info] token: BadToken
How to convert the binary request body to String? I want to see the body and headers in clear text.
I had a conversation with the http4s team on gitter and found the response. since gitter talk is not returned by google I am putting the answer here
val loggedReq = req.copy(body = request.body.observe(scalaz.stream.io.stdOutBytes))
println(loggedReq)
this prints all the headers. If we do something with the loggedReq then we get the entire body which is posted
loggedReq.as[String].run
Related
I am building a scala application. Within the application, we are making a call to an external service, and fetching the data.
When I am hitting the endpoint of this external service using the
postman, I am getting the complete data, around 9000 lines of JSON
data, within 9 seconds.
But when I am hitting the same endpoint through my scala application, I am getting a 200 OK response, but getting the below error:
[WARN] [06/09/2022 18:05:45.765] [default-akka.actor.default-dispatcher-9] [default/Pool(shared->http://ad-manager-api-production.ap-south-1.elasticbeanstalk.com:80)] [4 (WaitingForResponseEntitySubscription)] Response entity was not subscribed after 100 seconds. Make sure to read the response entity body or call `discardBytes()` on it. GET /admin/campaigns Empty -> 200 OK Chunked
I read about it and found that we can set response-entity-subscription-timeout property to a higher value. I set it to about 100 seconds, but this does not seem to help.
My Code:
private val sendAndReceive = customSendAndReceive.getOrElse(HttpClientUtils.singleRequest)
.
.
.
def getActiveCampaigns: GetActiveCampaigns = () => {
val request = HttpRequest(
uri = s"$endpoint/admin/campaigns?status=PUBLISHED", // includes both PUBLISHED_READY and PUBLISHED_PAUSED
method = HttpMethods.GET,
headers = heathers
)
sendAndReceive(request).timed(getActiveCampaignsTimer).flatMap {
case HttpResponse(StatusCodes.OK, _, entity, _) =>
Unmarshal(entity).to[List[CampaignListDetailsDto]]
case response#HttpResponse(_, _, _, _) =>
response.discardEntityBytes()
Future.failed(new RuntimeException(s"Ad manager service exception: $response"))
case response =>
log.error(s"Error calling ad manager service: $response")
response.discardEntityBytes()
Future.failed(new RuntimeException(s"Ad manager service exception: $response"))
}
}
.
.
.
def getCampaignSpendData(getActiveCampaigns: GetActiveCampaigns, getCampaignTotalSpend: GetCampaignTotalSpend)(implicit ec: ExecutionContext): GetCampaignsSpendData = () => {
getActiveCampaigns()
.andThen {
case Failure(t) => log.error("Failed to fetch ads from ad manager", t)
}
.flatMap {
campaignList => Future.sequence(campaignList.map(campaign => budgetSpendPercentage(getCampaignTotalSpend)(campaign)))
}
}
Questions
What does this error exactly mean? Is it that it is able to connect to the endpoint but not able to get the complete data from it before the connection is closed/reset?
How can we address this issue?
Currently running tests for my REST API which:
takes an endpoint from the user
using that endpoint, grabs info from a server
sends it to another server to be translated
then proceeds to jsonify the data.
I've written a series of automated tests running and I cannot get one to pass - the test that actually identifies the content of the response. I've tried including several variations of what the test is expecting but I feel it's the actual implementation that's the issue. Here's the expected API response from the client request:
{ "name": "random_character", "description": "Translated description of requested character is output here" }
Here is the testing class inside my test_main.py:
class Test_functions(unittest.TestCase):
# checking if response of 200 is returned
def test_healthcheck_PokeAPI(self):
manualtest = app.test_client(self)
response = manualtest.get("/pokemon/")
status_code = response.status_code
self.assertEqual(status_code, 200)
# the status code should be a redirect i.e. 308; so I made a separate test for this
def test_healthcheck_ShakesprAPI(self):
manualtest = app.test_client(self)
response = manualtest.get("/pokemon/charizard")
self.assertEqual(response.status_code, 308)
def test_response_content(self):
manualtest = app.test_client(self)
response = manualtest.get("/pokemon/charizard")
self.assertEqual(response.content_type,
'application/json') <<<< this test is failing
def test_trans_shakespeare_response(self):
manualtest = app.test_client(self)
response = manualtest.get("/pokemon/charizard")
self.assertFalse(b"doth" in response.data)
Traceback:
AssertionError: 'text/html; charset=utf-8' != 'application/json' - text/html; charset=utf-8 + application/json
Any help would be greatly appreciated
I' am using groovy to consume a POST Rest api : here is my code :
import groovyx.net.http.RESTClient
#Grab (group = 'org.codehaus.groovy.modules.http-builder', module = 'http-builder', version = '0.7.1')
def url = "https://poc-ser.tst.be/"
def client = new RESTClient(url)
client.ignoreSSLIssues()
client.auth.basic(login,pswd)
client.post(
path: "osidoc/api/rest/production/documents?id=46",
contentType:'application/json',
headers: [Accept: 'application/json' , Authorization: 'Authorization']
)
But I always get error " error groovyx.net.http.ResponseParseException: OK caused by: groovy.json.JsonException: Unable to determine the current character, " Knowing that the response should be "application/msword" type (i.e : the response should be a word doc)
EDIT
I tried to change the Accept to "application/octet-stream' but it showed me an other error "406Invalid Accept header. Only XML and JSON are supported.`"
I need to create a Groovy post build script in Jenkins and I need to make a request without using any 3rd party libraries as those can't be referenced from Jenkins.
I tried something like this:
def connection = new URL( "https://query.yahooapis.com/v1/public/yql?q=" +
URLEncoder.encode(
"select wind from weather.forecast where woeid in " + "(select woeid from geo.places(1) where text='chicago, il')",
'UTF-8' ) )
.openConnection() as HttpURLConnection
// set some headers
connection.setRequestProperty( 'User-Agent', 'groovy-2.4.4' )
connection.setRequestProperty( 'Accept', 'application/json' )
// get the response code - automatically sends the request
println connection.responseCode + ": " + connection.inputStream.text
but I also need to pass a JSON in the POST request and I'm not sure how I can do that. Any suggestion appreciated.
Executing POST request is pretty similar to a GET one, for example:
import groovy.json.JsonSlurper
// POST example
try {
def body = '{"id": 120}'
def http = new URL("http://localhost:8080/your/target/url").openConnection() as HttpURLConnection
http.setRequestMethod('POST')
http.setDoOutput(true)
http.setRequestProperty("Accept", 'application/json')
http.setRequestProperty("Content-Type", 'application/json')
http.outputStream.write(body.getBytes("UTF-8"))
http.connect()
def response = [:]
if (http.responseCode == 200) {
response = new JsonSlurper().parseText(http.inputStream.getText('UTF-8'))
} else {
response = new JsonSlurper().parseText(http.errorStream.getText('UTF-8'))
}
println "response: ${response}"
} catch (Exception e) {
// handle exception, e.g. Host unreachable, timeout etc.
}
There are two main differences comparing to GET request example:
You have to set HTTP method to POST
http.setRequestMethod('POST')
You write your POST body to outputStream:
http.outputStream.write(body.getBytes("UTF-8"))
where body might be a JSON represented as string:
def body = '{"id": 120}'
Eventually it's good practice to check what HTTP status code returned: in case of e.g. HTTP 200 OK you will get your response from inputStream while in case of any error like 404, 500 etc. you will get your error response body from errorStream.
I call an API to get a zip file response. The API responds correctly but I am unable to get the byte array from response because the future that should complete on getting the ByteString never completes:
val authorization = akka.http.javadsl.model.headers.Authorization.basic("xxxxx", "xxxxxx")
val query = Map("fed" -> "xxxx", "trd" -> "yyy", "id" -> "zzz")
val request = HttpRequest(HttpMethods.GET, Uri("https://xxxx.yyyy.com/ggg/ttt.php").withQuery(Query(params = query))).addHeader(authorization)
val responseFut = http.singleRequest(request)
responseFut1.map(response => {
println("*******************************")
println(response)
response.status match {
case akka.http.javadsl.model.StatusCodes.OK => {
println("^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^" + response._3)
val entityFut = response.entity.toStrict(60.seconds)
val byteStringFut = entityFut.flatMap(entity => {
entity.dataBytes.runFold(ByteString.empty)(_ ++ _)
})
println("#############################")
try {
byteStringFut.map(x => {
//this never prints =======================================problem
println("----------------------------" + x.toArray[Byte])
})
}catch{
case e: Exception => println("Error: " + e)
}
}
case _ => {}
}
})
If I print out the response this is what it looks like:
*******************************
HttpResponse(200 OK,List(Date: Fri, 08 Sep 2017 20:58:43 GMT, Server: Apache/2.4.18 (Ubuntu), Content-Disposition: attachment; filename="xxxxx.zip", Pragma: public, Cache-Contr
ol: public, must-revalidate, Content-Transfer-Encoding: binary),HttpEntity.Chunked(application/x-zip),HttpProtocol(HTTP/1.1))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^HttpEntity.Chunked(application/x-zip)
#############################
So response is coming back fine, but I still cannot get the binary data for the zip file.
We use akka-http in other places to call APIs that return json response and this approach seems to work fine there.
Why doesnt it work here? What am I doing wrong?
Any advice is appreciated.
Thanks.
Update
Adding byteStringFut.failed.foreach(println(_)) shows this exception: akka.http.scaladsl.model.EntityStreamException: HTTP chunk size exceeds the configured limit of 1048576 bytes
It looks like something went wrong and an exception was thrown in the async computation. You can check the exception by inspecting Future in the following way:
byteStringFut.failed.foreach(println)