Forbidden error while retrieving details of cluster using <publicServiceEndpointURL> with REST kubernetes API - kubernetes

Trying to retrieve details of the kubernetes cluster like namespaces and pod details using kubernetes API
following doc
API:
<publicServiceEndpointURL>/api/v1/namespaces
Headers:
Authorization: bearer <id_token>
<id_token> - An IAM token generated.
will get an certificate error on postman with SSL certificate enabled else throwing 403 Forbidden error
Error: unable to verify the first certificate
Result with disabling SSL certificate verification.
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "namespaces is forbidden: User \"system:anonymous\" cannot list resource \"namespaces\" in API group \"\" at the cluster scope",
"reason": "Forbidden",
"details": {
"kind": "namespaces"
},
"code": 403
}
Tried with curl and will result in same error
curl -k <publicServiceEndpointURL>/api/v1/namespaces -H "Authorization: Bearer <token>"
Error on chrome with API call
net::ERR_CERT_AUTHORITY_INVALID
How will i able to access this API?

Do you have the right permissions to list namespaces in the cluster? If you log in as the same user via the CLI, for example, can you run kubectl get namespaces? It looks like a permissions error. The user would need IBM Cloud IAM Reader service role (which gives you RBAC view role) for all namespaces in the cluster.

Related

CloudFront to API Gateway request returns 403: "The request signature we calculated does not match the signature you provided."

I have an API Gateway fronted by CloudFront. The API Gateway has a regional endpoint with api key disabled. An Authorization header must be sent to the regional endpoint or the endpoint returns "Missing Authentication Token" as expected.
Using the same request on the CloudFront endpoint returns the following 403 Forbidden error:
{
"message": "The request signature we calculated does not match the signature you
provided. Check your AWS Secret Access Key and signing method. Consult the service
documentation for details.\n\nThe Canonical String for this request should have been\n'POST
// sensitive data here...
}
The Auth token is created from an AWS signature. The signature originates from an IAM role that allows invocation on the endpoint: "Action": "execute-api:Invoke"
Any ideas on why CloudFront isn't able to use these credentials to hit the API Gateway endpoint?
In summary,
"Postman w/ Authorization header -> API Gateway endpoint" works.
"Postman w/ Authorization header -> CloudFront -> API Gateway endpoint" returns the above 403.
UPDATE: Adding information on how I obtain the signature.
IAM Role:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:us-west-2:{ACCOUNT}:{ENDPOINT}",
"Effect": "Allow"
}
]
}
AccessKey, SecretKey, Session Token are obtained in CloudShell:
$ aws sts assume-role --role-arn arn:aws:iam::{ACCOUNT}:{ROLE} --role-session-name {SESSION_NAME}
These 3 keys are then used in Postman's Authorization tab. I select "AWS Signature" type and provide the AccessKey, SecretKey, and SessionToken.
From here, I can hit the API Gateway endpoint and receive 200 response. With the same request and headers, hitting the CloudFront endpoint results in the 403.
UPDATE #2: Adding information on CloudFront configuration.
The distribution behavior for the API GW origin is using the CachingOptimized policy. Its also allowing all HTTP methods.

Keycloak most basic installation giving 401 Status Code on AKS

Installing keycloak on minikube works fine after following https://www.keycloak.org/getting-started/getting-started-kube.
But when I try on Azure Kubernetes Service; I am getting following response on browser for URL https://keycloak.10.18.80.36.nip.io/ OR https://keycloak.10.18.80.36.nip.io/admin
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "Unauthorized",
"reason": "Unauthorized",
"code": 401
}
I am not even getting screen to put in username and password. Help is appreciated. Also, when I was accessing from browser it was giving SSL error; so I proceed with Not secure. I am using IP of AKS "API server address"
I was finally able to solve it to great extent.
Use helm chart. Good part of using helm charts is that it also creates postgres which takes care of persistence layer. Otherwise whenever pod is deleted all users will go away.
helm repo add azure-marketplace https://marketplace.azurecr.io/helm/v1/repo
helm install keycloak-helm azure-marketplace/keycloak
Now check the Service ( Load Balancer ) external IP and use it to access on browser. This was the key part. as we don't have minikube so we cannot use $ minikube ip
On browser you can login using following credentials:
Username: user
echo Password: $(kubectl get secret --namespace default keycloak-helm -o jsonpath="{.data.admin-password}" | base64 --decode)
apHc7vK5vL

Authentication with Openshift API running on IBM cloud failed with 401 unauthorized error

I'm trying to build some application to manage my OpenShift cluster on IBM cloud and the first step is to authenticate against both IBM cloud and the OpenShift cluster.
https://cloud.ibm.com/docs/openshift?topic=openshift-cs_api_install#kube_api
I followed the steps describe in the above link, and successfully obtained all the tokens including 'access_token', 'id_token' and 'refresh_token'. Among them the 'id_token' is supposed to be used to authenticate against the OpenShift API.
With the access_token I can visit IBM cloud API successfully, like obtaining account, cluster information.
However, when I use the id_token to call OpenShift API, it failed with the following error. It happened even for the '/version' api, which can be accessed without providing a bearer token.
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "Unauthorized",
"reason": "Unauthorized",
"code": 401
}
I can verify that my account have correct service roles assigned as described here, and I can see corresponding roles with 'ibm' prefix assigned in OpenShift web portal as well.
Can anyone please verify that the instructions in the first link above is still valid or have any clue about what might have been wrong?
[Update]
To help troubleshooting, I paste a sample of tokens here, this is what I get for the step3 in the 'Working with your cluster by using the Kubernetes API' section in the link, it is a bit lengthy:
{
"access_token": "eyJraWQiOiIyMDIxMDIxOTE4MzUiLCJhbGciOiJSUzI1NiJ9.eyJpYW1faWQiOiJJQk1pZC0yNzAwMDU1WERHIiwiaWQiOiJJQk1pZC0yNzAwMDU1WERHIiwicmVhbG1pZCI6IklCTWlkIiwianRpIjoiMDY1OWI5MjktMDE1Zi00MDg0LTgwZWMtYmFhZjBhYTBkNDQ4IiwiaWRlbnRpZmllciI6IjI3MDAwNTVYREciLCJnaXZlbl9uYW1lIjoi6Iic5a2QIiwiZmFtaWx5X25hbWUiOiLpmYgiLCJuYW1lIjoi6Iic5a2QIOmZiCIsImVtYWlsIjoicmFmb3VsQDE2My5jb20iLCJzdWIiOiJjaHN6Y2hlbkBjbi5pYm0uY29tIiwiYXV0aG4iOnsic3ViIjoiY2hzemNoZW5AY24uaWJtLmNvbSIsImlhbV9pZCI6IklCTWlkLTI3MDAwNTVYREciLCJuYW1lIjoi6Iic5a2QIOmZiCIsImdpdmVuX25hbWUiOiLoiJzlrZAiLCJmYW1pbHlfbmFtZSI6IumZiCIsImVtYWlsIjoicmFmb3VsQDE2My5jb20ifSwiYWNjb3VudCI6eyJ2YWxpZCI6dHJ1ZSwiYnNzIjoiOWM5NzI1YmQxM2VhNDU2Nzg4YWMwZWU3OGQ4NjQ2ZTEiLCJpbXNfdXNlcl9pZCI6Ijg4NzM1NzYiLCJmcm96ZW4iOnRydWUsImltcyI6IjM0NjU1MiJ9LCJpYXQiOjE2MTQyNTU5ODYsImV4cCI6MTYxNDI1OTU4NiwiaXNzIjoiaHR0cHM6Ly9pYW0uY2xvdWQuaWJtLmNvbS9pZGVudGl0eSIsImdyYW50X3R5cGUiOiJ1cm46aWJtOnBhcmFtczpvYXV0aDpncmFudC10eXBlOmFwaWtleSIsInNjb3BlIjoiaWJtIG9wZW5pZCBjb250YWluZXJzLWt1YmVybmV0ZXMiLCJjbGllbnRfaWQiOiJrdWJlIiwiYWNyIjoxLCJhbXIiOlsicHdkIl0sInN1Yl85Yzk3MjViZDEzZWE0NTY3ODhhYzBlZTc4ZDg2NDZlMSI6ImNoc3pjaGVuQGNuLmlibS5jb20iLCJpYW1faWRfOWM5NzI1YmQxM2VhNDU2Nzg4YWMwZWU3OGQ4NjQ2ZTEiOiJJQk1pZC0yNzAwMDU1WERHIiwicmVhbG1lZF9zdWJfOWM5NzI1YmQxM2VhNDU2Nzg4YWMwZWU3OGQ4NjQ2ZTEiOiJJQk1pZC1jaHN6Y2hlbkBjbi5pYm0uY29tIn0.Rm3F0UKz9Aq3-1xXMmkFi0UkENIvQUkRo6qhtWaG3LKBH5HHsZbAQeJUhKqXYbI643nj2ssDP2U50BVv-6zbpfmyVncP5Z5Dmi620mi2QesduRQaH1XlC-l7KuF3uT0hJ_9FSD-0Wqi5ph0pkKxHJ-BmLkHC-4F0NByiUtwIpwyTpthuzwC251XZsQ9Ya8gzCxHB9DFb3tzOF3cupVVZmc2mMJbv4JuTSnP00H5rOT4yIzeI0Lqm6LhDpMRJ4P8glmIxmU6fag42P94pFNf3jEzIZGl49NINiWXlKbAleij3vSouobtYvrBmxWQF4KpuwKPEI-bMf1zpsHPYBHWidg",
"id_token": "eyJraWQiOiIyMDIxMDIxOTE4MzUiLCJhbGciOiJSUzI1NiJ9.eyJpYW1faWQiOiJJQk1pZC0yNzAwMDU1WERHIiwiaXNzIjoiaHR0cHM6Ly9pYW0uY2xvdWQuaWJtLmNvbS9pZGVudGl0eSIsInN1YiI6ImNoc3pjaGVuQGNuLmlibS5jb20iLCJhdWQiOiJrdWJlIiwiZ2l2ZW5fbmFtZSI6IuiInOWtkCIsImZhbWlseV9uYW1lIjoi6ZmIIiwibmFtZSI6IuiInOWtkCDpmYgiLCJlbWFpbCI6InJhZm91bEAxNjMuY29tIiwiZXhwIjoxNjE0MjU5NTg2LCJzY29wZSI6ImlibSBvcGVuaWQgY29udGFpbmVycy1rdWJlcm5ldGVzIiwiaWF0IjoxNjE0MjU1OTg2LCJhdXRobiI6eyJzdWIiOiJjaHN6Y2hlbkBjbi5pYm0uY29tIiwiaWFtX2lkIjoiSUJNaWQtMjcwMDA1NVhERyIsIm5hbWUiOiLoiJzlrZAg6ZmIIiwiZ2l2ZW5fbmFtZSI6IuiInOWtkCIsImZhbWlseV9uYW1lIjoi6ZmIIiwiZW1haWwiOiJyYWZvdWxAMTYzLmNvbSJ9LCJzdWJfOWM5NzI1YmQxM2VhNDU2Nzg4YWMwZWU3OGQ4NjQ2ZTEiOiJjaHN6Y2hlbkBjbi5pYm0uY29tIiwiaWFtX2lkXzljOTcyNWJkMTNlYTQ1Njc4OGFjMGVlNzhkODY0NmUxIjoiSUJNaWQtMjcwMDA1NVhERyIsInJlYWxtZWRfc3ViXzljOTcyNWJkMTNlYTQ1Njc4OGFjMGVlNzhkODY0NmUxIjoiSUJNaWQtY2hzemNoZW5AY24uaWJtLmNvbSIsImdyb3Vwc185Yzk3MjViZDEzZWE0NTY3ODhhYzBlZTc4ZDg2NDZlMSI6WyJkZXZvcHMtYWRtaW4tdnBuLW9ubHkiLCJkZXZvcHMtZGVmYXVsdC12cG4tb25seSIsIklQV0MgQWRtaW4iXX0.Y42KUJRGgZA9OV164GAKSF0W5rRNGf3x32YXrAo5UvKhpOK0k4r_hwZU5BZhI2y3t-UqM7lNOIxexpft2Zmc9ApQ6BlVN-iN1jcfBzxmrUPMObpc1-vDrAc9Sq84J8nYzy1Rk32ydFHeb3V2iDhJn14_NOnXwhuz9EFkSg0uUZHugTAPx5A-VcdrehceX0yOqAOfX5EzTtmHoI8-JQbfNt8pyBSJs8Eoag7_mtfNgx13bP_-M8W7tltCSHhPEO46gUurPFkvasHggConPQ_oBw3ANAvY8tDfivrGmdiR2Q-uc4SnFAjOgC77YskDLskBcOeehhBvxwDkyufztzqM6w",
"refresh_token": "OKDsw87zCujUXCmb4LZ3-DFQN7lUa0ejdqau_fL3Voms7M7DaKYgO07gZW29VQbcwdGc3z8jrQjjf_4gOutKyRCZ6LyEiSEKTZQ6Kovwqji02Puxu3fzIFB9f8-a1hMlkTtP4u32_FTCmOZA6ARvzxEyRX36CtQEzSVz-zVMsvPxdgyztUEWPTtvbr7aPn4eq209OzTGzTyPCBFR-N0gVp2tKLbIrGmyi_vgC-6xLRvR2nWGJsUwaaBjXwvICeCBY3qRJ90VyP1krBSHa72f1XJWpvLnBWHN8qo1dfPknHvknlEZ3kMUA87KZkynkgiVifhRq90oNAKYHhKJ4XRs2tyz05zW5a8qEhgoIVsslUzDLLNU1btRF_3g587dKckPzEav3BgQlCik4im8gIC74HFGZOz4P7z9QKLJHQY7ElDillH8pLRjW8Dx0yZvn8Yo5rSqJSj0zUmJxNZMUNEpF_DTQhHCePNOWu1_1q4o5cIb_Mv-mGMMVwrVUsJYUyaeV9O5cWl58eWlHQxS3SbuAjsBrzfSdcrIyFe5aQViyL_sL1-o54xFrMJPC3prPD25TS4vUOwAy7tc9r1AGZG00YUGaxPwzKcOWBI4DqksIiEKPOtcm3k0y24TuwRPa0AK-9jfYAzkx3rciBYGKbq1WOFjX-p6LH67ayxVUJcQcjSMe-35LZnsHQtc0VOxNHjJKdJiHsKOYEDY1Nz0k4zGZr1EZ6j7w4tLpBXP9ThC8hReiihWDmld9lzFdLwKZPF7jl4u03a2WQZ6j-wMHvLtOBcLDiKwEaeWaGp8v_YS3j4iGqkcAytf7z_-toD1O3ZHtIUlbe6H64IAVPKadN1Y1SD49Ouk1fk8xDFr7HQ4RuDTLfZnLGzC4vvzysCmJEX837Wjf2f9WdirEaKxoSlDDJKilt--20Ota-5CTimD8u0SttC6CD1Glj8bbAS8ddCAfVirDJty7FW3eyALvAHifKqzRa1kBDPHb305q91oSWYdzBKIlTinN9BAXDc3ZccVkWM6Y3VgUzh2iQwM0lKadts7OMwqhLDk7rukAXHRUpKxy-85rUf-a0oz41s69PXdQteoh559vEb0uyrq0kOnI1RnuJ7MaEGDC25Kfezumo0snwYRmQhXMPMeKkxBKxs9ZydKxxcp1qtLwFyHA6MhZuXRpZM9Qse9mqovNdHHOhAQIZu3J7HJusuVdg3SJhZkTH__gXpCc2hBeOpR0rPc6qZm7z2nU5pJQ2XgzH2TUm6psA",
"ims_user_id": 8873576,
"token_type": "Bearer",
"expires_in": 3600,
"expiration": 1614259586,
"refresh_token_expiration": 1616847976,
"scope": "ibm openid containers-kubernetes"
}
In addition, the following approach works but the token is obtained through the OpenShift web console, and thus cannot be obtained programmatically(at least I don't see how),
"Authorization: Bearer sha256~6V_OvZ5OoV8vnHF33Es5qsloAY-iXkLQ8dfl_Nsyn94"
Thanks!
You can not and should not send the ID-Token to get access to APIs, its only meant to be used by the client who did the initial authentication. It also typically have a very short lifetime (like 5 minutes in some implementation).
The only purpose of the ID-token is basically o create the local user session.
On the page you refer to it says at the end:
ID token: Every IAM ID token that is issued via the CLI expires after
one hour. When the ID token expires, the refresh token is sent to the
token provider to refresh the ID token. Your authentication is
refreshed, and you can continue to run commands against your cluster.
It sounds like they mean the access token. In openID connect you don't renew your ID-token (what I am aware of)
Have been busy in the past few days, I will share how I solved this problem here. In fact it didn't address the original issue, but is another way to achieve the goal.
So it turned out that there was another doc regarding how the access token can be obtained(Yes, as mentioned by #Tore Nestenius it should be an access token instead of an id token). The token described here is actually the same as what one would get through the Openshift web console. And basically it has nothing to do with the previous link I shared in the question.

How to reach openapi interface for minikube?

According to Kubernetes documentation, you can ping an http endpoint for a K8s cluster to operate it instead of using e.g. kubectl. It also says that there is an openApi interface available at /openapi/v2. I'm running minikube on MacOS and would like to check it out. Does anyone know if that is possible?
I tried:
minikube list service # Shows 'kubernetes' as a name
minikube list kubernetes
... which opens the browser to 127.0.0.1:51377. However, when I try going to https://127.0.0.1:51377/openapi/v2 I get the message:
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "forbidden: User \"system:anonymous\" cannot get path \"/openapi/v2\"",
"reason": "Forbidden",
"details": {},
"code": 403
}
... suggesting that I need some sort of authorization solution. Suggestions?
This worked for me:
# Proxy minikube to localhost on arbitrary port:
kubectl proxy --port=12345
# Now swagger.json is available at localhost:12345/openapi/v2
# Save to /tmp/temp/json and serve with e.g. docker swagger-ui container
curl localhost:12345/openapi/v2 > /tmp/temp.json
docker run -it -p 9999:8080 -e SWAGGER_JSON=/var/specs/temp.json -v /tmp/temp.json:/var/specs/temp.json swaggerapi/swagger-ui
# Open browser to localhost:9999

Kubernetes REST API - Unauthorized

I have following Kubernetes REST API request
GET https://theserver/api/v1/pods?includeUninitialized=true
and included following HTTP Headers in the request:
Authorization: Basic ***************
Accept: application/json, */*
User-Agent: kubectl.exe/v1.13.0 (windows/amd64) kubernetes/ddf47ac
Result is the following error
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "Unauthorized",
"reason": "Unauthorized",
"code": 401
}
I already tried the following:
Analog kubectl request "kubectl describe pods --all-namespaces" works fine (but I need REST)
Used "--v=12" parameter as in newkind101's comment to Kubernetes REST API to see the underlying REST API calls - looks same as mine
Read Kubernetes Documentation - but I failed to find detail information which HTTP headers or/and HTTP body values to send to authenticate properly (few thing I could retrieve from curl sample calls in that docs)
I read Access Kubernetes API using REST APIs but want to understand my issue before I use a framework like GoDaddy
kubectl seems to do a bit more than I can see with the "--v=12" parameter. This bit is likely connected to things in ".kube/config" file. Still I don't know what exactly and where to put in my HTTP request.
As far as I remember Basic authentication method is disabled by default and needs to be enabled by adding --basic-auth-file flag to the API server configuration, therefore all requests to REST API are identified as anonymous user and might be resulted in 401 Unauthorized error.