I am able to do the following using CLI.
// Get role id
vault read auth/approle/role/irma-role/role-id
Key Value
role_id b1e32157-8309-d5a2-02c9-657fc05977dc
// Get secret id
vault write -force auth/approle/role/irma-role/secret-id
Key Value
secret_id 3cac8255-6f2d-e62f-fd0e-20bfe69a2dfe
secret_id_accessor 8f811a56-d2c4-460c-8955-f2a18d8a2fdd
// Login using role id and secret id
vault write auth/approle/login role_id="b1e32157-8309-d5a2-02c9-657fc05977dc" secret_id="ea34790f-f8c4-5527-3951-4e51b8b6e620"
The above works great. But I need help in figuring out how to do using API. For example, how to I get the X-Vault-Token that the API is looking for while retrieving secret id?
$ curl \
--header "X-Vault-Token: ..." \
--request POST \
http://127.0.0.1:8200/v1/auth/approle/role/application1/secret-id
Related
I would assume this to be straight forward but I can't find it in the docs.
The following curl command:
curl \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $ACCESS_TOKEN" \
'https://$URL/auth/realms/$REALM/protocol/openid-connect/userinfo'
results in:
{
"sub": "8182...415",
"email_verified": true,
"name": "n.a. n.a.",
"groups": [],
"preferred_username": "foo#example.com",
"given_name": "n.a.",
"family_name": "n.a.",
"email": "foo#example.com"
}
How do I get the custom attributes for a user?
You can get the user attributes with the get users endpoint from Admin Rest API:
GET /{realm}/users
with the query parameters, exact=true and username.
Step-by-Step:
You can get that information using the Keycloak Admin REST API; to call that API, you need an access token from a user with the proper permissions. For now, I will be using the admin user from the master realm:
curl https://${KEYCLOAK_HOST}/auth/realms/master/protocol/openid-connect/token \
-d "client_id=admin-cli" \
-d "username=$ADMIN_NAME" \
-d "password=$ADMIN_PASSWORD" \
-d "grant_type=password"
You will get a JSON response with the admin's token. Extract the value of property access_token from that response. Let us save it in the variable $ACCESS_TOKEN for later reference.
To get the user attributes from your realm $REALM_NAME:
curl -X GET https://${KEYCLOAK_HOST}/auth/admin/realms/${REALM_NAME}/users/?username=${USERNAME}&exact=true \
-H "Content-Type: application/json" \
-H "Authorization: bearer $ACCESS_TOKEN"
From the response extract the user attributes for example as follows:
jq -r .[].attributes
To retrieve custom user attributes via the userinfo endpoint you need to create a protocol Mapper for the client used to authenticate the user.
That mapper can also be created with the Keycloak Admin rest API. For a more detailed answer on how to create Protocol Mappers for user-attributes (including for the old and new Keycloak APIs) please have a look at the this SO answer.
Or you can do it via Keycloak Admin UI as follows, in the Keycloak go to:
Select your realm
Go to clients
Select the appropriate client for your use-case
(For the OLD Keycloak UI)
Go to Mappers
Click Create
Select Mapper Type as User Attribute
Fill up the field User Attribute with your custom user attribute
Set to be added to the userinfo endpoint
Fill up the remaining fields, accordingly
Click on Save
(For the NEW Keycloak UI)
Go to the tab Client Scopes
Click on the scope -dedicated (e.g., test-dedicated in my example)
Click on Configure a new mapper (or Add Mapper > By configuration if you have already created mappers before for this client)
Select User Attribute
Fill up the field User Attribute with your custom user attribute
Set to be added to the userinfo endpoint
Fill up the remaining fields, accordingly
Click on Save
This is enough to enabled your custom user attribute to be retrieved from the userinfo endpoint
I want to connect service principal with certificate to authorize using pyspark. I could see the code in scala in this link - https://github.com/Azure/azure-event-hubs-spark/blob/master/docs/use-aad-authentication-to-connect-eventhubs.md
I have client _id and tenant_id and certificate details. Could some please share me the code in pyspark for same?
You add the Azure AD service principal to the Azure Databricks workspace using the SCIM API 2.0. Authentication using Pyspark isn't available.
To authenticate using service principal, you need to follow below steps:
As you already have clientID and tenantID, so the service principal ID already created.
Create the Azure Databricks personal access token
You’ll use an Azure Databricks personal access token (PAT) to authenticate against the Databricks REST API. To create a PAT that can be used to make API requests:
Go to your Azure Databricks workspace.
Click the user icon in the top-right corner of the screen and click User Settings.
Click Access Tokens > Generate New Token.
Copy and save the token value.
Add the service principal to the Azure Databricks workspace
You add the Azure AD service principal to a workspace using the SCIM API 2.0. You must also give the service principal permission to launch automated job clusters. You can grant this through the allow-cluster-create permission. Open a terminal and run the following command to add the service principal and grant the required permissions:
curl -X POST 'https://<per-workspace-url>/api/2.0/preview/scim/v2/ServicePrincipals' \
--header 'Content-Type: application/scim+json' \
--header 'Authorization: Bearer <personal-access-token>' \
--data-raw '{
"schemas":[
"urn:ietf:params:scim:schemas:core:2.0:ServicePrincipal"
],
"applicationId":"<application-id>",
"displayName": "test-sp",
"entitlements":[
{
"value":"allow-cluster-create"
}
]
}'
Replace with the unique per-workspace URL for your Azure Databricks workspace.
Replace with the Azure Databricks personal access token.
Replace with the Application (client) ID for the Azure AD application registration.
Create an Azure Key Vault-backed secret scope in Azure Databricks
Secret scopes provide secure storage and management of secrets. You’ll store the secret associated with the service principal in a secret scope. You can store secrets in a Azure Databricks secret scope or an Azure Key Vault-backed secret scope. These instructions describe the Azure Key Vault-backed option:
Create an Azure Key Vault instance in the Azure portal.
Create the Azure Databricks secret scope backed by the Azure Key Vault instance.
Step 1: Create an Azure Key Vault instance
In the Azure portal, select Key Vaults > + Add and give the key vault a name.
Click Review + create.
After validation completes, click Create .
After creating the key vault, go to the Properties page for the new key vault.
Copy and save the Vault URI and Resource ID.
Step 2: Create An Azure Key Vault-backed secret scope
Azure Databricks resources can reference secrets stored in an Azure key vault by creating a Key Vault-backed secret scope. To create the Azure Databricks secret scope:
Go to the Azure Databricks Create Secret Scope page at https:///#secrets/createScope. Replace per-workspace-url with the unique per-workspace URL for your Azure Databricks workspace.
Enter a Scope Name.
Enter the Vault URI and Resource ID values for the Azure key vault you created in Step 1: Create an Azure Key Vault instance.
Click Create.
Save the client secret in Azure Key Vault
In the Azure portal, go to the Key vaults service.
Select the key vault created in Step 1: Create an Azure Key Vault instance.
Under Settings > Secrets, click Generate/Import.
Select the Manual upload option and enter the client secret in the Value field.
Click Create.
Grant the service principal read access to the secret scope
You’ve created a secret scope and stored the service principal’s client secret in that scope. Now you’ll give the service principal access to read the secret from the secret scope.
Open a terminal and run the following command:
curl -X POST 'https://<per-workspace-url/api/2.0/secrets/acls/put' \
--header 'Authorization: Bearer <personal-access-token>' \
--header 'Content-Type: application/json' \
--data-raw '{
"scope": "<scope-name>",
"principal": "<application-id>",
"permission": "READ"
}'
Replace with the unique per-workspace URL for your Azure Databricks workspace.
Replace with the Azure Databricks personal access token.
Replace with the name of the Azure Databricks secret scope that contains the client secret.
Replace with the Application (client) ID for the Azure AD application registration.
Create a job in Azure Databricks and configure the cluster to read secrets from the secret scope
You’re now ready to create a job that can run as the new service principal. You’ll use a notebook created in the Azure Databricks UI and add the configuration to allow the job cluster to retrieve the service principal’s secret.
Go to your Azure Databricks landing page and select Create Blank Notebook. Give your notebook a name and select SQL as the default language.
Enter SELECT 1 in the first cell of the notebook. This is a simple command that just displays 1 if it succeeds. If you have granted your service principal access to particular files or paths in Azure Data Lake Storage Gen 2, you can read from those paths instead.
Go to Workflows and click the + Create Job button. Give the job a name, click Select Notebook, and select the notebook you just created.
Click Edit next to the Cluster information.
On the Configure Cluster page, click Advanced Options.
On the Spark tab, enter the following Spark Config:
fs.azure.account.auth.type.acmeadls.dfs.core.windows.net OAuth
fs.azure.account.oauth.provider.type.acmeadls.dfs.core.windows.net org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider
fs.azure.account.oauth2.client.id.acmeadls.dfs.core.windows.net <application-id>
fs.azure.account.oauth2.client.secret.acmeadls.dfs.core.windows.net {{secrets/<secret-scope-name>/<secret-name>}}
fs.azure.account.oauth2.client.endpoint.acmeadls.dfs.core.windows.net https://login.microsoftonline.com/<directory-id>/oauth2/token
Replace with the name of the Azure Databricks secret scope that contains the client secret.
Replace with the Application (client) ID for the Azure AD application registration.
Replace with the name associated with the client secret value in the secret scope.
Replace with the Directory (tenant) ID for the Azure AD application registration.
Transfer ownership of the job to the service principal
A job can have exactly one owner, so you’ll need to transfer ownership of the job from yourself to the service principal. To ensure that other users can manage the job, you can also grant Can Manage permissions to a group. In this example, we use the Permissions API to set these permissions.
Open a terminal and run the following command:
curl -X PUT 'https://<per-workspace-url>/api/2.0/permissions/jobs/<job-id>' \
--header 'Authorization: Bearer <personal-access-token>' \
--header 'Content-Type: application/json' \
--data-raw '{
"access_control_list": [
{
"service_principal_name": "<application-id>",
"permission_level": "IS_OWNER"
},
{
"group_name": "admins",
"permission_level": "CAN_MANAGE"
}
]
}'
Replace with the unique per-workspace URL for your Azure Databricks workspace.
Replace with the Azure Databricks personal access token.
Replace with the Application (client) ID for the Azure AD application registration.
The job will also need read permissions to the notebook. Run the following command to grant the required permissions:
curl -X PUT 'https://<per-workspace-url>/api/2.0/permissions/notebooks/<notebook-id>' \
--header 'Authorization: Bearer <personal-access-token>' \
--header 'Content-Type: application/json' \
--data-raw '{
"access_control_list": [
{
"service_principal_name": "<application-id>",
"permission_level": "CAN_READ"
}
]
}'
Replace with the unique per-workspace URL for your Azure Databricks workspace.
Replace with the ID of the notebook associated with the job. To find the ID, go to the notebook in the Azure Databricks workspace and look for the numeric ID that follows notebook/ in the notebook’s URL.
Replace with the Azure Databricks personal access token.
Replace with the Application (client) ID for the Azure AD application registration.
You can now test the job. You run jobs with a service principal the same way you run jobs as a user, either through the UI, API, or CLI.
I am very new to keycloak and I want to know is it possible to create public client then authorize client and get access token. Then use that access token and create UMA tickets to resource server and check if client has rights to access resoucrces using same authorization
I uses this turtorial
https://gruchalski.com/posts/2020-09-05-introduction-to-keycloak-authorization-services/
And it worked if you reauthorized to resource server directly.
But I want to use one authorisation to check rights to resources which could belong to different resource servers
I get access token using this code:
export access_token=`curl --silent -u public-client:1 \
-k -d "grant_type=password&username=${USER_NAME}&password=${USER_PASSWORD}&scope=email profile" \
-H "Content-Type:application/x-www-form-urlencoded" \
${KEYCLOAK_TOKEN_URL} | jq '.access_token' -r`
Then I try to authorise same user to different client:
curl --silent -X POST \
${KEYCLOAK_TOKEN_URL} \
-H "Authorization: Bearer ${access_token}" \
--data "grant_type=urn:ietf:params:oauth:grant-type:uma-ticket" \
--data "audience=resource-server-1"
But I get "Client does not support permissions"
Maybe someone knows what I need to configure for this to work or even if this can work?
For what I've reading you need two clients. One public and the other confidential.
Then in your client configured as confidential set all your fine-grained authz then in your audience param you set the client id previously configured as confidential.
My HashiCorp vault instance is runnning properly on CentOS7. I enabled AppRole authentication, created a policy and a role, enabled secret engine and created a secret for a client application.
I can retrieve the secret data using root CLI but I can't figure out how to get secret data from HTTP API with my application role using curl. I tried a few endpoint combinations without success. Retrieving the client token works, but I can't get secret data itself.
I wonder if the API endpoint is correct or if there is another setting in play.
Authentication method
vault auth enable approle
Policy
# File: my_app /etc/vault/my_app.hcl
path "kv/data/foo/*" {
capabilities = ["read", "list"]
}
# Command line
vault policy write my_app /etc/vault/my_app.hcl
Role
vault write auth/approle/role/my_app policies="my_app"
Secret creation
vault kv put kv/data/foo/user#domain.tld password=1234
API call token request
curl --request POST --data '{"role_id": "xxxxxxxxxxxxxxxxx", "secret_id": "xxxxxxxxxxxxxxxxxxxx"}' http://127.0.0.1:8200/v1/auth/approle/login | jq
Result: Token is properly retrieved
API call for secret data request
export VAULT_CLIENT_TOKEN=XXXXXXX
curl --header "X-Vault-Token: $VAULT_CLIENT_TOKEN" --request GET "http://127.0.0.1:8200/v1/kv/data/foo/user#domain.tld"
Result : No secret data retrieved
Output:
{"errors":[]}
CLI call for secret data
vault kv get -field=password kv/data/foo/user#domain.tld
Output:
1234
Global settings
vault secrets list
Path Type Accessor Description
---- ---- -------- -----------
cubbyhole/ cubbyhole cubbyhole_xxxxxxxx per-token private secret storage
identity/ identity identity_xxxxxxxx identity store
kv/ kv kv_xxxxxxxx n/a
sys/ system system_xxxxxxxx system endpoints used for control, policy and debugging
kubectl config set-credentials USER_NAME \
--auth-provider=oidc \
--auth-provider-arg=idp-issuer-url=( issuer url ) \
--auth-provider-arg=client-id=( your client id ) \
--auth-provider-arg=client-secret=( your client secret ) \
--auth-provider-arg=refresh-token=( your refresh token ) \
--auth-provider-arg=idp-certificate-authority=( path to your ca certificate ) \
--auth-provider-arg=id-token=( your id_token ) \
--auth-provider-arg=extra-scopes=( comma separated list of scopes to add to "openid email profile", optional )
Is this sufficient to configure openid-connect with kubernetes?
Can someone tell me what are the value for
1. issuer url
2. refresh token
3. path to ca certificate
4. extra-scopes
5. id-token
Also just to confirm, i would like to know if client secret and client ID are the same as the google credentials created.
Install the k8s-oidc-helper tool. To this pass the google credential (JSON file). This will then open the browser to provide a secret code.
Copy and paste it to get the required OIDC paramters.