I'm trying to generate access token to access google APIs without the google consent screen. It will enable us to implement code which can automate certain actions in GCP like stopping a VM and lots more. I prefer to use dotnet core for this implementation.
I except to generate a bearer auth token through which I can access Google rest APIs without any environment setup or Google OAuth consent screen.
I have found answer for this question and it will be useful for the community members who are working on GCP or google APIs.
First you have to download json credentials for your gcp account, please refer to this documentation Download service account keys
Please refer to this git repo gcp-auth , it's dotnet core class library which contains static method to GenerateJwt, ExchangeTokenAsync and GetAccessTokenAsync.
Let me explain each method for better clarity.
GenerateJwt - this method generates a signed JWT token from PrivateKey, PrivateKeyID, ServiceAccountEmail which are present in downloaded json from previous step and scope which will be different for different api endpoint like we have https://www.googleapis.com/auth/compute for google apis under compute.
ExchangeTokenAsync - this method need the generated JWT from previous step to generate the actual access token from google which will be valid to access google apis under that scope.
GetAccessTokenAsync - this is combination of above two methods, here you've pass parameters same as passed in GenerateJwt.
Please let me know if it requires any further explanation.
I am trying to create a test suite in Azure DevOps by using the Rest API. But here I am not sure how to provide the security details.
They have mentioned below details in docs. I am passing it in Postman like this. I tried giving my credentials in client ID and Client Secret as well. Please guide
{"$id":"1","innerException":null,"message":"A potentially dangerous Request.Path value was detected from the client (&).","typeName":"System.Web.HttpException, System.Web","typeKey":"HttpException","errorCode":0,"eventId":0}
There are several ways to call Azure DevOps REST API, in your question, it uses Oauth 2.0 flow to authenticate, you could follow this doc to get the token and call the API.
In your case, if it is not necessary to use Oauth 2.0 flow to do the authentication, actually I recommend you to use PAT(personal access token) to call the API, just generate one and use it in the postman, refer to this doc.
Can the namespace preferences and program preferences be set via REST API calls? If yes, what is the syntax for it?
Generally in Cloud Data Fusion, when we intend to perform the action on GCP side, like create/delete/restart etc. instance, it's feasible to use domestic Google Cloud API, giving the opportunity to interact with a service endpoint via JSON/HTTP calls interface as described in Google Cloud API design document.
Dedicated to Data Fusion you can follow the Cloud Data Fusion REST API reference document, nicely explaining the methods for composing REST API HTTP calls to manage Data Fusion instances, moreover every method description from the documentation contains Google API Explorer sub-panel, to get handy experience building JSON request on a live data.
Said above, I assume your initial question is related more to CDAP REST API, as it includes the methods for pure CDAP instance metadata/namespaces/application configuration.
From the user perspective your workflow might be the following:
Identify the CDAP API endpoint as explained in this guideline;
Compose an HTTP PUT/GET request relevant to Data Fusion
Namespace/Metadata/Preferences/Configuration
object via CDAP RESTful API.
Yes of course! You have two methods.
The first method is creating it from the platform. Follow the steps below:
Open your data fusion instance
Go to System Admin => Configuration => Make HTTP calls
To create a namespace, submit an HTTP PUT request:
PUT /v3/namespaces/<namespace-id>
Link of CDAP: CDAP
The second method is using terraform.
I am working on an API project and it is going to be exposed to members of our system. We need to secure our API using API key.
When the API is invoked, we need to know which member is invoking the API.
So we need to map the key with the member some how.
I am wondering what is the best way to manage the keys. Following are the options we are aware of
For each member, keep the API keys in our backend system and manage the key in our backend and once key is changed, manually update it in API gateway
Whenever key is changed in backend system, invoke the AWS API gateway rest service to update the key in API gateway
Any other options( not looking or OAuth2 like complex key management solutions)
Api Gateway API Keys are not used for authentication but identification. Read the last pargraph here. Judging from your question that you need to know which member invokes the API Gateway etc. means that you most likely need to implement an authentication system and the best candidate here is AWS Cognito. Use the right tool for the job and don't try to reinvent the wheel.
The Problem
MS Azure provides an extensive REST API. However, there is a significant amount of complexity when trying to get that API to work. From outdated and incomplete documentation to simple examples not working, performing what should be an easy task is instead nightmarish.
The Task
What are the exact, precise, detailed steps necessary to list the available VMs for someone who has logged in using Oauth2? For instance, this can be done using the azure-cli.
azure vm list
What are the steps to accomplish the same thing using REST and Oauth2?
The Requirements
The answer must not use Visual Studio, PowerShell, C#, an SDK, or any other such tool to accomplish this task. Only the portal is allowed for setup, and only Oauth2 is allowed for authorization, and only the REST API is allowed for actual information retrieval.
The answer must not simply link to external sources, although external references are encouraged for completeness.
The answer may assume the user has an Azure account. It must include information about creating the Oauth2 client, credentials, and any step necessary to get the appropriate token.
The answer must be detailed.
The answer must provide examples. Images, too are strongly encouraged.
The answer should include information about possible errors, their meanings, and what too look for to fix them.
First, we can find this rest API in azure resource portal. It is the same with Azure CLI command azure vm list.
I have tested it on my local with http request. here is my tested screenshot:
Request URL: https://management.azure.com/subscriptions/<subscription ID>/providers/Microsoft.Compute/virtualMachines?api-version=2016-03-30
Header:Authorization: bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI.....
So It is very import if we get the access token. The following demonstrate us how to get the access token.
Get Token(POST):
Request URL: https://login.microsoftonline.com/<tenant id>/oauth2/token
Body: grant_type=client_credentials&client_id=<client id>&resource=https%3A%2F%2Fmanagement.core.windows.net%2F&client_secret=<client secret>
Here is my screenshot in fildder:
We need to get client id and client secret in azure ad application. For how to regist an application in Azure AD. Please have a look at this article: https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal.
Please note:
1) we need to add "Windows Azure Service Management API" in portal "Required permissions" like the following screenshot:
2) We need assign "Contributor" for this service principal. click subscription-> Access Control-> click Add -> click "Select a Role" -> click Contributor->click Add User-> Find the application you created above-> click OK.
Overview
Making requests against the Azure Rest API is a bit more complicated than perhaps you would think at first glance. In particular, there are a number of esoteric and not-so-helpful error messages you may run into while getting the nobs tweaked just right.
Introduction and Terms
Setting up The Application
Getting the access_token
Making the API request
Common errors thus far
Introduction and Terms
One of the pieces of this process that can make it so confusing and difficult is the terminology. Until you understand that, knowing how to deal with errors is very difficult. We'll go over some of the more common ones here.
Subscription - This is basically what you'd expect. It refers to the Microsoft Azure Services subscription. It basically acts as the top-level umbrella for the organization.
Tenant - This is like a sub-organization, maybe a department or group. There can be multiple tenants under a single subscription.
User - As expected, a user is a single individual. Users are scoped to tenants.
Application - The Application is the software program trying to use the API. It must be registered and configured to do so.
Service Principal - This is essentially The Application. It is the entity making API Service requests.
Setting Up the Application
Although you might not guess it, this is probably the most complicated part of the process. Let's start by creating The Application in the portal.
Create The Application
Follow this click path: Azure Active Directory -> App Registrations -> New
There should be a form for application creation, with the following fields:
Name
This is simple the name of The Application. When authenticating, it will be shown to users. For the purposes of this "tutorial", we'll call it API Tutorial. This can be edited after creation.
Application Type
The type of The Application. For our purposes, we should choose "Web app/API". This cannot be edited after creation.
Sign-on URL
The is the redirect that will be used if we go the "authorization_code" route for sign on. This can be useful as the response will include an "id_token". We'll get into that a bit later. For now, let's make this http://123AzureApiTutorial.com/code. This can be edited after creation.
Once the Application has been created, you should see a property, Application ID. This is the client_id used in the OAuth2 flow. Take a note of its value.
Create the Client Secret
The OAuth2 flow requires a client secret value for authentication.
To generate it, follow this click path: Azure Active Directory -> App Registrations -> API Tutorial -> All Settings -> Keys
Enter the key description: API Tutorial Key, and the Duration: In 1 year.
Click Save. This will generate the Key Value. You must copy the value here and save it somewhere. You will not have another opportunity to do so.
This value is the client_secret in the OAuth2 flow.
Add the correct permissions
To get to the permissions, follow this click path: Azure Active Directory -> App Registrations -> API Tutorial -> All Settings -> Required Permissions -> Add
Here you will see the list of possible APIs. The one we care about for Azure is Windows Azure Service Management API. There is currently only one permission: Access Azure Service Management as organization users (preview). Select it, click Select, and then Done.
Getting the access_token
The access_token is what allows us to make requests against the API. There are two primary ways to do this. I suggest reading about both before trying to implement them.
Authorization Code
The authorization code is a two-step process. First we obtain the authorization code, and then we use that to get the access_token. A benefit of this route is that we get back an id_token as well, with a variety of useful claims like the user's name, email address, etc.
The request format is as follows: (GET) https://login.microsoftonline.com/<tenant-id>/oauth2/authorize?client_id=<client-id>&scope=api&redirect_uri=<redirect-uri>&response_type=code&prompt=consent. Let's go over the parameters here really quick.
Tenant ID
This can be obtained be using the click path Azure Active Directory -> Properties and copy the Directory ID. This is, in fact the Tenant ID value. It just has a different name to help with the overall confusion.
Client ID
This is the Application ID we retrieved previously.
Scope
This is the scope of the code. We just want to use the API.
Redirect URI
This is the sign-on URL you specified when creating API Tutorial. After the user logs in, they will be redirected to this URL with a "code" parameter in the query string.
Response Type
This is what we want the response to be. We want an authorization code, so we just use the value code.
Prompt
This specifies whether or not to prompt the user to consent to the permissions. If we did not have this, and changed permissions, the request would just unexpectedly fail. Very frustrating. But it can be removed once permission has been granted as long as you don't change the permissions. If the application is accessing an API that requires admin permission, this value should be admin_consent.
Alright, so once we shoot off this request we will be redirected to the login page. We login, accept the permissions, and then we should be redirected to here: http://123AzureApiTutorial.com/code?code=SOME_REALLY_LONG_STRING_OF_CHARACTERS. That string of characters is the code.
Getting the Access Token
Next, we take the code and use it to get the access_token. To do so, we need to make another request.
(POST) https://login.microsoftonline.com/<tenant-id>/oauth2/token
In addition to the url, we need to add parameters. These should be consistent with the content type application/x-www-form-urlencoded. This means they are submitted as form parameters. They are as follows:
client_id
This is again the client id (Application ID) we already have.
client_secret
This is the Application Key we generated earlier. I hope you saved it! If not, go back to that step and generate another one.
code
This is the value of the code we just received: SOME_REALLY_LONG_STRING_OF_CHARACTERS.
`grant_type
Because we're going the authorization code route, this value should be authorization_code
redirect_uri
This is the redirect uri we specified for the API Tutorial. The value from our example should be http://123AzureApiTutorial.com/code.
resource
This is very important. It is the resource API we want to access. For the Azure API, this value is https://management.azure.com.
Our response will be a json object with a variety of fields. Of these, the one we care about is access_token. Yay!
Client Credentials
This methodology skips getting the code (and thus needing the redirect_uri) at the expense of not getting an id_token.
The request is the same as in the Getting the Access Token section, with a few small differences.
We do not need to specify redirect_uri.
The value of grant_type should change to client_credentials.
Alright, we have an access_token! Now we're cooking!
Making the API request
With all the prep work thus far, this is the easiest part of the process.
The API URL we are requesting against is:
https://management.azure.com/subscriptions/<subscription-id>/providers/Microsoft.Compute/virtualMachines?api-version=2016-03-30
Add the following header to the request:
Authorization: Bearer <access-token>. Yes, the access_token value must be prefaced with the word "Bearer" in the header value.
"But wait!" You exclaim. "How do I get the subscription id?"
Excellent question! To find it through the portal, click Subscriptions -> -> Overview and copy the Subscription Id value.
Use that value, and run the request. You should see all the vms listed!
Common errors thus far
InvalidAuthenticationToken
When making the API request, you get an error response that says something like this: InvalidAuthenticationToken: The access token is invalid. This means you haven't added the API permissions to the API Tutorial. Go back to the Add the correct permissions step and double-check you have the right permissions. Also, when requesting the token make sure you use the prompt=consent, otherwise the you will not be prompted with the new permissions and the token will fail.
InvalidAuthenticationTokenTenant
Make sure that the tenant-id used when requesting the token belongs to the subscriber used when making the API call.