I am trying to add a REST API to my powerBI report as a data source. The API uses Basic authentication. However, I don't want to keep my token in an unencrypted form in the query editor.
Currently, I am able to make it work by using the following in Advanced Query Editor with Anonymous authentication:
let
token = "apiKey",
options = [Headers=[Authorization="Basic " & token]],
Source = Json.Document(Web.Contents("https://api-url/api/v1/application", options))
in
Source
However, I am trying to use a method, so that I can put my apiKey in the data source credentials, and eventually keep this value as private.
Either as basic auth
Or as web API key
Related
Hello Pentaho Experts,
I am attempting to make a REST API call through REST Client in Pentaho. I have a Api key for authentication. I tried it in Postman, and it works perfectly fine, but Pentaho throws 403 status code. Below is the postman screenshot:
My Pentaho transformation contains two steps. I am passing URL through "Generate Rows" step and then adding Key and Value in the Header:
Generate Rows:
Rest Client (General):
Rest Client (Header):
Any idea what I might be missing here? Expected output is JSON.
I can't speak to the use of passing key/value in Header for authentication, but what has worked for me is to use the Authentication tab to input my credentials or passing an Authentication header with a bearer token.
Authentication Tab:
I use this method to generate and return a bearer token from the API. The token is then used as the authentication method in subsequent steps. The "body" includes the scope (i.e.: "reports:read") and grant type ("client_credentials"). The "header" contains the value "application/x-www-form-urlencoded".
Authentication Header:
Once a token has been returned from the API, this can be provided in an "authorization" header. Prepend "bearer" to the token value and pass this in the header tab.
I would also recommend looking at the Headers being passed in Postman to ensure you are not missing anything else that might be happening in your call.
I'm trying to download a CSV file from a website in Data Factory using the HTTP connector as my source linked service in a copy activity. It's basically a web call to a url that looks like https://www.mywebsite.org/api/entityname.csv?fields=:all&paging=false.
The website uses basic authentication. I have manually tested by using the url in a browser and entering the credentials, and everything works fine. I have used the REST connector in a copy activity to download the data as a JSON file (same url, just without the ".csv" in there), and that works fine. But there is something about the authentication in the HTTP connector that is different and causing issues. When I try to execute my copy activity, it downloads a csv file that contains the HTML for the login page on the source website.
While searching, I did come across this Github issue on the docs that suggests that the basic auth header is not initially sent and that may be causing an issue.
As I have it now, the authentication is defined in the linked service. I'm hoping that maybe I can add something to the Additional Headers or Request Body properties of the source in my copy activity to make this work, but I haven't found the right thing yet.
Suggestions of things to try or code samples of a working copy activity using the HTTP connector and basic auth would be much appreciated.
The HTTP connector expects the API to return a 401 Unauthorized response after the initial request. It then responds with the basic auth credentials. If the API doesn't do this, it won't use the credentials provided in the HTTP linked service.
If that is the case, go to the copy activity source, and in the additional headers property add Authorization: Basic followed by the base64 encoded string of username:password. It should look something like this (where the string at the end is the encoded username:password):
Authorization: Basic ZxN0b2njFasdfkVEH1fU2GM=`
It's best if that isn't hard coded into the copy activity but is retrieved from Key Vault and passed as secure input to the copy activity.
I suggest you try to use the REST connector instead of the HTTP one. It supports Basic as authentication type and I have verified it using a test endpoint on HTTPbin.org
Above is the configuration for the REST linked service. Once you have created a dataset connected to this linked service you can include it in you copy activity.
Once the pipeline executes the content of the REST response will be saved in the specified file.
The aim is to connect to a public REST api using ADF. It's my first stab at sending requests to a REST api in ADF. It is the Companies House ('CH') governement website's API in England.
I have created an account and obtained a key. Apparently, it is basic authentication and the user name is the API key and password will be ignored (CH note on authentication)
I want to explore the contents of the 'Search all' API (CH note on Search All) and want to copy the results to Blob Storage.
I therefore set the linked service to use REST as below, the obfuscated User Name is the key I obtained from CH, the password is jsut the key repeated as their documentation states they ignore the password:
[
I then have added a REST dataset referencing this linked service:
And the testing of the connection works fine.
Problems then arise in the copy data task, I am getting an error when previewing and also when I attempt a copy to blob of 'Invalid Authorization Header':
I'd be grateful for pointers on where I'm going wrong.
I can't reproduce your Auth error but i notice that you want to add some parameters with your GET request in the Request Body.
I think you need to add parameters in relativeUrl property:
A relative URL to the resource that contains the data. When this
property isn't specified, only the URL that's specified in the linked
service definition is used. The HTTP connector copies data from the
combined URL: [URL specified in linked service]/[relative URL
specified in dataset].
Also i suggest you checking the correct REST API format of Search Api you are using.There is no other special features in the ADF REST connector. Just make sure the GET request works locally and duplicate it.
I would like to know how to property transfer from one REST API to another REST API?
I worked on the property transfer with the JSON format, but I need to know how to transfer the one REST API response property and use it in the header?
let me explain this using the below scenario:
Admin get the User object by API
Save apiKey from the user object
Call the action on user behalf(By set Authorization Header to Basic with the user apiKey)
To accomplish these steps in soapUI You have to:
Define apiKey in Custom Properties in TestCase
Call resource that returns a User object
Save apiKey property to TestCase.apiKey via property transfer or Groovy Step
Reuse TestCase.apiKey in your nest REST Request step by expanding it via Basic ${=(testCase.getPropertyValue("apiKey")+":").bytes.encodeBase64()}
Hope that helps
I need to do a POST request to get the access token from Google Analytics using a service account.
I need to bypass manual authorisation in a browser, so I have used a service account for which I have all the details, private_key, client_id, etc
https://www.googleapis.com/oauth2/v4/token&grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion={PRIVATEKEY}
If I do the above, I receive not found.
Can anyone help?
Cheers
The Server Side Authorization Embed API demo has an example of doing this. The example uses the Google APIs client library for Python, but, as #DalmTo mentions in their comment, you could look at what it's doing under the hood to replicate the requests.
Here's what the code that gets the access token looks like:
import json
from oauth2client.client import SignedJwtAssertionCredentials
# The scope for the OAuth2 request.
SCOPE = 'https://www.googleapis.com/auth/analytics.readonly'
# The location of the key file with the key data.
KEY_FILEPATH = 'path/to/json-key.json'
# Load the key file's private data.
with open(KEY_FILEPATH) as key_file:
_key_data = json.load(key_file)
# Construct a credentials objects from the key data and OAuth2 scope.
_credentials = SignedJwtAssertionCredentials(
_key_data['client_email'], _key_data['private_key'], SCOPE)
# Defines a method to get an access token from the credentials object.
# The access token is automatically refreshed if it has expired.
def get_access_token():
return _credentials.get_access_token().access_token