HttpClient Get works when Fiddler is on and fails with 403 when fiddler is not running - httpclient

I have seen many posts but none of the solution is helping me. I have a simple get to one of the Amazon new webservices. The client is HttpClient.
When fiddler is on it passes and gets me result. When fiddler is not running it fails. Since it was not failing when fiddler was on, i installed another tool "Http Debugger" to see the failure and i could not make any difference in both the request (The pass and the failure). Bothe of them are given below
Failure Capture:
GET /sellers/v1/marketplaceParticipations HTTP/1.1
Accept: application/json
x-amz-access-token: Atza|IwEBIN3hOVtNi1xM47txtHcqXi5A3C960AypB7pkWYCxEo7lNiL9EFR-1b_EoD6PQ8lzAXgM4zujF0OBv0NS7sYQ9bWqPMDhHFt8kgvdmlmk3==jrinknrO6PYlZgmFLRGn1Hzmvgldmnj4973bjkfnbkldlcvld vc0BmuqKHUreonrWQxFO49u0yoIiNHVzSxHP0Wo4nWKW5pdd5Fj73gYxnZQQeYF5EAy8lKDCLtndTnJCdlrv5Kk8JK8iFD_H7h3FF5H4gNyTx3uIHxMaU8OkLz_IigsCTNQHwljnubhQlR9aK0J6lRbb0QfOQ4BAT_e1GOKDkShu-U5OdchdF5qNUkKU
user-agent: MSolution/1.0.0.0
X-Amz-Date: 20201201T224620Z
Authorization: AWS4-HMAC-SHA256 Credential=AKIA5SU6JNBJKDAQB76813QI6V/20201201/eu-west-1/execute-api/aws4_request, SignedHeaders=accept;host;user-agent;x-amz-access-token;x-amz-date, Signature=71dd12ee0eaf33cd142dwr242424e91cb5c4bfd6fd4f46d929d
Host: sellingpartnerapi-eu.amazon.com
Connection: Keep-Alive
The error is Forbiden (Just this)
The success capture is (Fiddler running):
GET /sellers/v1/marketplaceParticipations HTTP/1.1
Accept: application/json
x-amz-access-token: Atza|IwEBIMfdZDDaca8HrDGIPft-HQs3Vzi75I4Bk9iNKfsHTkfwsfsfcsvcsaP86DqKkoZE37TiDr3XvmD_vdvavcUE9TzdXhf2jjuULL04keBHI_XYrnTnhXaCPE0gUAc8HvIiW7OXSERz_3RlS9R-nu2lTo_jqzaz0mbUaa-evavaVAVLauh2Ue7Io8pE1tThRTcqM60igPcrBViAUptTAsq-IL5ZT7hOfbNJTJ31GeN8e8IzjkWfe9n4l7B799VM1bJnC-D_alZ2J0HHj4cBNjd3RzAEvavav3fGWkW5iH2_MZ3IyaxYnslvSzNH4h8tvay87OywkkxVUKIn
user-agent: MSolution/1.0.0.0
X-Amz-Date: 20201201T224820Z
Authorization: AWS4-HMAC-SHA256 Credential=AKIAYWBDAQ7XYIPQI6V/20201201/eu-west-1/execute-api/aws4_request, SignedHeaders=accept;host;user-agent;x-amz-access-token;x-amz-date, Signature=7e4db1c114219546848eaffnvclknslnlcvs63c9e0af50edc3cdbe7231c9b
Host: sellingpartnerapi-eu.amazon.com
I have used:
ServicePointManager.ServerCertificateValidationCallback +=
(senderSP, certificate, chain, sslPolicyErrors) => true;
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12 | SecurityProtocolType.Ssl3;
Not sure what is causing this but has anyone seen this ?
The same call through "RestSharp" works, but i donot wnat to use rest sharp.
Shankar

Related

sapui5 OData.V4.oDataModel with Teiid 11.2 / Wildfly

I am trying to use SAPUI5 in a SPA to display data from a TEIID/Wildfly ODataV4 service. When SAPUI5 ODATA V4 data model is bound to the service, I run into several errors. I thereby connect via proxy (grunt-connect-proxy2) to the odata service. Basic Auth works. The metadata file above the marked service folder in the attached image (screenshot1) is the actual metadata file of the service which, as you can see, is loaded correctly. So no CORS issue or authorization issue.
Screenshot1
The issue seems to be related to an CSRF Token request as far as I understand. Seems that Teiid/Wildfly is not answering the CSRF Token fetch request. Is there a way to configure Wildfly to answer the request or alternatively a way to disable CSRF requests for the odata V4 model? I have seen such an option in the constructor of the odata V2 model. How could a working configuration look like?
The following is a screenshot from the browser log:
I observed a further issue, from which I do not know if it is related to the previous one (there is also a X-CSRF-Token: Fetch involved), or if a have something more missing somewhere. The second issue happens when I use an aggregation binding to bind a odata collection to a sapui5 list. The response looks like
Request URL: http://localhost:9001/odata4/svc/my_nutri_diary/$batch
Request Method: POST Status Code: 406 Not Acceptable Remote Address:
[::1]:9001 Referrer Policy: no-referrer-when-downgrade Response
Headersview source access-control-allow-credentials: true
access-control-allow-origin: http://localhost:9001 cache-control:
no-cache, no-store, must-revalidate connection: close
content-encoding: gzip content-length: 125 content-type:
application/json;odata.metadata=minimal date: Mon, 12 Nov 2018
20:04:30 GMT expires: 0 odata-version: 4.0 pragma: no-cache server:
WildFly/11 x-powered-by: Undertow/1 Request Headersview source Accept:
multipart/mixed Accept-Encoding: gzip, deflate, br Accept-Language: de
Authorization: Basic SU1TVXNlcjpJTVM0Zm9ydW0l Connection: keep-alive
Content-Length: 329 Content-Type: multipart/mixed;
boundary=batch_id-1542053070786-11 Cookie: sidebar_collapsed=false;
cycle_analytics_help_dismissed=1;
__utmz=111872281.1539128843.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=111872281.767670437.1539128843.1541866362.1541870562.42 DNT: 1 Host: localhost:9001 MIME-Version: 1.0 OData-MaxVersion: 4.0
OData-Version: 4.0 Origin: http://localhost:9001 Referer:
http://localhost:9001/ User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS
11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko)
Version/11.0 Mobile/15A372 Safari/604.1 X-CSRF-Token: Fetch
X-Requested-With: XMLHttpRequest Request Payload
--batch_id-1542053070786-11 Content-Type:application/http Content-Transfer-Encoding:binary GET Profile?$skip=0&$top=100 HTTP/1.1
Accept:application/json;odata.metadata=minimal;IEEE754Compatible=true
Accept-Language:de
Content-Type:application/json;charset=UTF-8;IEEE754Compatible=true
--batch_id-1542053070786-11--
Thanks for your advice!
Best regards,
Christoph
Further note regarding search for a workaround: As I am currently searching for a workaround to be able to use the odata.v4 model, I found the following blog post:
https://blogs.sap.com/2015/08/05/disable-csrf-token-for-odata-calls-using-sap-netweaver-gateway/
However, the approach does not seem to work for the odata.v4 model as it has a different interface. There is no setHeaders() function to set custom headers on the datamodel. I therefore tried to set the header up via
$.ajaxSetup({headers: {'X-Requested-With': 'X'}});
Unfortunately, this also does not work. If someone has the odata.v4 Model running with TEIID or Olingo v4 it would be great if he could give me a feedback on how he had worked around this issue.

URLSession on Linux giving different result than on iOS

I've got a simple class making a HTTP POST request (to a Neo4j server, but that shouldn't matter) using URLSession, and an integration test that tests that it succeeds.
Through Xcode, via the iOS simulator, I can run this test a million times - it succeeds every time.
However, when I run swift test on Ubuntu Linux (I use the docker image provided by IBM) then I very often will get a test failure saying I got a 401 response.
Replicating that via cURL in the container does not fail at all, so I do not believe this to be a problem with my container.
I have added a packet dump (that I inspect via Charles Proxy) where our of 15 test runs, 7 test runs (and thus 7 requests) fail. All of the failed requests complain that no authentication header was supplied. And from the dump, that is correct, the dump does not contain the authentication header for those requests that fail. But why not? In fact, all of the header flags are different: a successful run has these headers:
POST /db/data/cypher HTTP/1.1
Host: 192.168.0.18:7474
Accept-Encoding: deflate, gzip
Authorization: Basic bmVvNGo6c3RhY2swdmVyRmxvdw==
Content-Type: application/json; charset=utf-8
Accept: application/json; charset=utf-8
Connection: keep-alive
User-Agent: urlsessionTestPackageTests.xctest (unknown version) curl/7.35.0
Content-Length: 135
while an unsuccessful one has these:
POST /db/data/cypher HTTP/1.1
Host: 192.168.0.18:7474
Accept: */*
Accept-Encoding: deflate, gzip
Connection: keep-alive
User-Agent: urlsessionTestPackageTests.xctest (unknown version) curl/7.35.0
Content-Length: 135
All the 200 results have the same headers, and all the 401 results have the same headers. Can you see anything in my code that should warrant such a random request?

How to get messages from ActiveMQ using REST requests

I am sending message to queue using following request:
POST http://localhost:8161/api/message/PR?type=queue
Host: localhost:8161
Authorization: Basic YWRtaW46YWRtaW4=
User-Agent: curl/7.48.0
Accept: */*
Content-Type: application/x-www-form-urlencoded
Content-Length: 12
Then I go to the admin UI and I see the queue and the message there.
Now the question, how can I retrieve this message?

AbsoluteURI support in Play Framework 2.1

As stated here:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html
To allow for transition to absoluteURIs in all requests in future versions of HTTP, all HTTP/1.1 servers MUST accept the absoluteURI form in requests, even though HTTP/1.1 clients will only generate them in requests to proxies.
I have client which sends POST-requests to my play-2.1.1 server. He sends it this way:
POST http://172.16.1.227:9000/A8%3aF9%3a4B%3a20%3a89%3a40/1089820966/ HTTP/1.1
Content-Length: 473
Content-Type: application/json
Date: Thu, 25 Apr 2013 15:44:43 GMT
Host: 172.16.1.227:9000
User-Agent: my-client
...some data...
All requests are rejected with "Action not found" error. The very same request which I send using curl is just fine and the only difference between them is curl send it with relative URI:
POST /A8%3aF9%3a4B%3a20%3a89%3a40/1089820966/ HTTP/1.1
Accept: */*
Content-Length: 593
Content-Type: application/json
Host: 172.16.1.227:9000
User-Agent: curl/7.30.0
I created the following simple workaround in Global.scala:
override def onRouteRequest(request: RequestHeader): Option[Handler] = {
if (request.path.startsWith("http://")) {
super.onRouteRequest(request.copy(
path = request.path.replace("http://"+request.host, "")
))
} else super.onRouteRequest(request)
}
And with this workaround all requests from my client are handled correctly.
So, is there more straightforward way to do it in Play Framework or thats the only way?
Thanks to #nraychaudhuri Play 2.2 supports absoluteURI-style request headers.
Here's the issue and pull request: https://github.com/playframework/playframework/pull/1060

Unable to get whole http message

I am using curl for sending a POST HTTP message to my server. At Server side I am opening a socket and reading the data by using following code
recv(socket_Fd, (void *)ucBuffer, (size_t)((sizeof(ucBuffer) - 1)), NULL);
I am able to get the header of the POST message but in message body I am getting only one line , rest are missing.
Data I am receiving at server end.
POST /info HTTP/1.1
User-Agent: curl/7.22.0 (i686-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
Host: 192.168.0.57:10000
Accept: */*
Content-Length: 356
Content-Type: application/x-www-form-urlencoded
Bhupesh Bhargava
In message header it's showing right content length but message body is missing. Any idea where I am doing wrong.
curl command I am using
curl --data-binary #/home/bhupesh/data_save2 http://192.168.0.57:10000/info
The curl command seems to be OK and if we go by the documentation here, the following should be true.
Data is posted in a similar manner as --data-ascii does, except that
newlines are preserved and conversions are never done.
So, this leaves us with the fact that there should be a problem in your Server implementation. It is not quite sure how you are getting the received stuff at the server, but you should be careful about sequencing what you receive by yourself. Here is an example how you could do that.