Azure Data Factory - Web activity with header value containing comma failing - azure-data-factory

We are trying to ingest data from Amazon Selling Partner API. We currently have an azure function that signs our request and returns the header values. The authorization header contains commas in it and the comma causes the activity to fail on the client side.
Here is an exampleof the authorization header:
AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20130524/us-east-1/s3/aws4_request,SignedHeaders=date;host;x-amz-content-sha256;x-amz-date;x-amz-storage-class,Signature=98ad721746da40c64f1a55b78f14c238d841ea1380cd77a1b5971af0ece108bd
To reproduce, create a new pipeline and add a web activity. Enter "http://www.google.com" for the url with the method GET. Add a header like above.
You should get the following error:
Error calling the endpoint 'http://www.google.com'. Response status code: 'NA - Unknown'. More details: Exception message: 'NA - Unknown [ClientSideException] The format of value 'AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20130524/us-east-1/s3/aws4_request,SignedHeaders=date;host;x-amz-content-sha256;x-amz-date;x-amz-storage-class,Signature=98ad721746da40c64f1a55b78f14c238d841ea1380cd77a1b5971af0ece108bd' is invalid.'.
Request didn't reach the server from the client. This could happen because of an underlying issue such as network connectivity, a DNS failure, a server certificate validation or a timeout.
Any work arounds for this?
Thanks,
Scott

If you are using Authorization as a header to get data make sure to use bearer in front of the Token.
Authorization Bearer < Access Token >
I tried to reproduce the same in my environment and got below results:
Output:
For more information refer this SO thread .

Related

Error getting authentication to Azure data factory

I need to hit the data factory pipeline with the REST call ...
So, I
Registered application in the Azure Active Directory.
Obtained Client_ID and client_Secret
Then I try to use Postman to get the authentication token and get an error, which I do not understand. Please, help. How can I get authentication token to my ADF using Postman ?
In body section, instead of raw, use - x-www-form-urlencoded as below and it should help resolve the issue.
Ideally, when you select x-www-form-urlencoded radio button in the Body section, it automatically sets the Content-Type=application/x-www-form-urlencoded under the Headers tab as shown below.
Please follow below steps for generating bearer token In ADF :
Step 1: Create a web activity to get the Access Token from ADF.
URL: https://login.microsoftonline.com/<Tenant ID>/oauth2/token
Method : POST
Body: grant_type=client_credentials&client_id=<client_id>&client_secret=<client_secret>&resource=https://management.azure.com/
Header: Content-Type:application/x-www-form-urlencoded
Step2: Create Set variable :
Add dynamic content -> #activity('Web1').output.access_token
Pipeline successfully executed and got the token

REST API calls through Pentaho Data Integration (Spoon)

Hello Pentaho Experts,
I am attempting to make a REST API call through REST Client in Pentaho. I have a Api key for authentication. I tried it in Postman, and it works perfectly fine, but Pentaho throws 403 status code. Below is the postman screenshot:
My Pentaho transformation contains two steps. I am passing URL through "Generate Rows" step and then adding Key and Value in the Header:
Generate Rows:
Rest Client (General):
Rest Client (Header):
Any idea what I might be missing here? Expected output is JSON.
I can't speak to the use of passing key/value in Header for authentication, but what has worked for me is to use the Authentication tab to input my credentials or passing an Authentication header with a bearer token.
Authentication Tab:
I use this method to generate and return a bearer token from the API. The token is then used as the authentication method in subsequent steps. The "body" includes the scope (i.e.: "reports:read") and grant type ("client_credentials"). The "header" contains the value "application/x-www-form-urlencoded".
Authentication Header:
Once a token has been returned from the API, this can be provided in an "authorization" header. Prepend "bearer" to the token value and pass this in the header tab.
I would also recommend looking at the Headers being passed in Postman to ensure you are not missing anything else that might be happening in your call.

How can I use REST API authentication in Mendix?

I have designed an API REST service (with Bonita) to which I can perfectly connect with Postman, with the following parameters:
By the way, the x-www-form-urlencoded option that is selected comes from the Content-type application/x-www-form-urlencoded header that is not displayed in my screenshot. The official Bonita specification states that this header is needed and I always get a 200-OK status code as an answer.
How can I specify an equivalent request with the body part in a Mendix Call REST service in a microflow? Here is what I have so far:
I guess the body part should be specified in the Request tab, but I just don't know how to do it properly. I always get the following error message for my connector, which means that, whatever I specify, the username is not taken into account:
An error has occurred while handling the request. [User 'Anonymous_69a378ed-bb56-4183-ae71-c9ead783db1f' with session id '5fefb6ad-XXXX-XXXX-XXXX-XXXXXXXXb34f' and roles 'Administrator']
I finally found that the proxy setting was the actual problem. It was set at the project scope and simply clicking on No proxy in the General tab did the trick! (both services are hosted on my local machine so far)
I just had to fill in the dedicated Authentication field in the HTTP Headers tab then, with the correct credentials, to eventually log in my Bonita service.

API Gateway HTTP integration, how do I pass the authorization header?

I'm in "Integration Request". I've set HTTP integration type and the Endpoint URL.
Below, I see "HTTP Headers, but if I try to add an header, there is a strange "mapped from" value I don't understand.
Setting an HTTP Proxy integration add a Mapping template I also don't understand.
I just need to pass the Authorization:xxxx and X-Auth-Username:xxxxx from the original request to the endpoint
Can't find an easy way...
I set HTTP proxy integration and added in the HTTP Headers (and deployed):
method.request.header.Authorization
method.request.header.X-Auth-Username
I also added as required Authoriaxtion and X-Auth-Username in Method request header as required.
But I get an error upon calling the api:
Authorization header requires 'Credential' parameter.
Authorization header requires 'Signature' parameter.
Authorization header requires 'SignedHeaders' parameter.
Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header
Hmmm maybe it's a "naming" problem?
So I changed "authorization" with MyAuthentication in the method request, and after in the integration Header, I mapped Authorization to
method.request.header.MyAuthentication
Now if I call my api with MyAuthorization in the header, I get "Missing Authentication Token", without hitting the api server
Click on 'Method Request' , expand 'HTTP Request Headers' and add a header Authorization . Now go back and click on 'Integration Request' , expand 'HTTP Headers' and add Header Name Authorization and 'Mapped from' method.request.header.Authorization . Basically for any header XYZ on 'Method Request' tab should have corresponding mapping on 'Integration Request' method.request.header.XYZ .
To Correctly use aws API Gateway as a pure http proxy and make it pass the Authorization header as-is to your backend API, you need to do two things:
In the Method Request: Make sure to add the Authorization header to the Http Request Headers section.
With this, AWS does not assume that the Authorization header is of your own implementation and the gateway does not expect it to contain AWS own format of multiple parameters such as SignedHeaders, Signature, X-Amz-Date , etc..
2. In the Integration Request: Make sure to select the Use HTTP Proxy Integration.
This makes sure that the request is being sent to the target as-is and thus the Authorization header not to be consumed and discarded as part of AWS own SigV4 authorization.
The following was correct...I was calling the original api path instead of te api gateway path....
I set HTTP proxy integration and added in the HTTP Headers (and deployed):
method.request.header.Authorization
method.request.header.X-Auth-Username

Why am I getting request method GET not supported?

I am using PostMan as a REST client to test this API method Cisco ACL Analysis API. specifically POST /acl/trace or getAClTracksStd (first go to Policy Analysis)
Here is my PostMan HTTP test call
Does anyone who is familiar with PostMan understand why I am getting this "Request method 'GET' is not supported" error from the server? I am making a POST HTTP request, not GET.(Selected from Drop down menu) It make more sense for me to get a input invalid parameter error or something.
Just to show that the endpoint url works, heres a HTTP test request that works
(same link, host->host API -> GET /host/{startIndex}/{recordsToReturn}
There's two issues that I'm seeing with your REST call. First, the error you're seeing is because the call needs to be preceded by "https://". Second, remove the interface IDs parameter and values. You should get a response with data after making these changes.
Your json looks erronuous (comma after the destIp) - and the server probably always responds with a default confusing error message in this case. (Postman is very well tested and it sends POST).