GitHub Enterprise authentication with Grafana not working - github

I try to set up GitHub authentication with Grafana. But I always receive this error:
{
"message": "API rate limit exceeded for 10.135.245.121. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)",
"documentation_url": "https://docs.github.com/enterprise/3.5/rest/overview/resources-in-the-rest-api#rate-limiting"
}
These are the logs after trying to sign in:
[ssm-user#ip-100-73-25-174 bin]$ sudo tail /var/log/grafana/grafana.log
logger=token t=2022-07-28T09:21:31.380404222Z level=debug msg=FeatureEnabled feature=accesscontrol.enforcement enabled=false licenseStatus=NotFound hasLicense=false hasValidLicense=false products="unsupported value type"
logger=token t=2022-07-28T09:21:31.380432476Z level=debug msg=FeatureEnabled feature=whitelabeling enabled=false licenseStatus=NotFound hasLicense=false hasValidLicense=false products="unsupported value type"
logger=live t=2022-07-28T09:21:31.429797544Z level=debug msg="Client disconnected" user=0 client=2cf47de6-e7b1-4d64-bf1d-0baab6ec9e97 reason=normal elapsed=2.707161792s
logger=token t=2022-07-28T09:21:31.679855249Z level=debug msg=FeatureEnabled feature=accesscontrol.enforcement enabled=false licenseStatus=NotFound hasLicense=false hasValidLicense=false products="unsupported value type"
logger=context traceID=00000000000000000000000000000000 userId=0 orgId=1 uname= t=2022-07-28T09:21:31.680567052Z level=info msg="Request Completed" method=GET path=/api/live/ws status=0 remote_addr=10.170.171.10 time_ms=0 duration=968.313µs size=0 referer= traceID=00000000000000000000000000000000
logger=live t=2022-07-28T09:21:31.714286816Z level=debug msg="Client connected" user=0 client=53a59f12-9c47-4c40-a57d-58a684058759
logger=token t=2022-07-28T09:21:33.705344746Z level=debug msg=FeatureEnabled feature=accesscontrol.enforcement enabled=false licenseStatus=NotFound hasLicense=false hasValidLicense=false products="unsupported value type"
logger=context traceID=00000000000000000000000000000000 userId=0 orgId=1 uname= t=2022-07-28T09:21:33.705978495Z level=info msg="Request Completed" method=GET path=/login/github status=302 remote_addr=10.170.171.10 time_ms=0 duration=899.188µs size=349 referer=https://users.tfe-nonprod.aws-cloud.axa-de.intraxa/grafana/login traceID=00000000000000000000000000000000
logger=live t=2022-07-28T09:21:34.018133518Z level=debug msg="Client disconnected" user=0 client=53a59f12-9c47-4c40-a57d-58a684058759 reason=normal elapsed=2.30377808s
logger=ngalert t=2022-07-28T09:21:37.530175528Z level=debug msg="alert rules fetched" count=0 disabled_orgs="unsupported value type"
My config:
# grafana.ini
[auth.github]
enabled = true
allow_sign_up = false
client_id = fea52015e8d3d4543276
client_secret = 3b53b35a0ea769e2e68e5769b6a4d142a40d023a
scopes = user:email,read:org
auth_url = https://github.axa.com/api/v3/login/oauth/authorize
token_url = https://github.axa.com/api/v3/login/oauth/access_token
api_url = https://github.axa.com/api/v3
team_ids =
allowed_organizations =
My ouath app config:
What I can also see is that the client secret is never used.

Related

ContainerApp Revision Failed when using dapr

I have created containerapp environment with one containerapp using Bicep template and here is a snippet of how I configured the environment
ingress: {
external: true
targetPort: 80
allowInsecure: false
transport:'http2'
traffic:[
{
latestRevision: true
weight: 100
}
]
}
registries: [
{
server: acr_login_server
username: acr_name
passwordSecretRef: 'myregistrypassword'
}
]
dapr: {
appId: containerapp_name
appPort: 80
appProtocol: 'http'
enabled: true
}
}
I am using http2 transport cause we expose grpc service as well, then when checking the revision, it shows failed and the logs shows that there is issue with dapr
time="2022-10-19T10:35:35.391746798Z" level=fatal msg="error loading configuration: rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: x509: certificate signed by unknown authority (possibly because of \\\"x509: ECDSA verification failure\\\" while trying to verify candidate authority certificate \\\"cluster.local\\\")\"" app_id=containerapp-a instance=containerapp-a--t1gheb2-77c44cf6c6-rxjwx scope=dapr.runtime type=log ver=1.8.4-msft-2

Why can't I login to Grafana with Keycloak integration?

I'm facing issue with Keycloak integration in Grafana:
With this grafana.ini:
instance_name = grafana
[log]
level = error
[server]
; domain = host.docker.internal
root_url = http://localhost:13000
enforce_domain = false
enable_gzip = true
[security]
admin_user = admin
admin_password = admin
[auth.generic_oauth]
name = OAuth
enabled = true
client_id = grafana
; client_secret = CLIENT_SECRET_FROM_KEYCLOAK
client_secret = <my client secret>
scopes = openid profile roles
; email_attribute_name = email:primary
auth_url = http://<keycloak IP>/auth/realms/mcs/protocol/openid-connect/auth
token_url = http://<keycloak IP>/auth/realms/mcs/protocol/openid-connect/token
api_url = http://<keycloak IP>/auth/realms/mcs/protocol/openid-connect/userinfo
allow_sign_up = false
disable_login_form = true
oauth_auto_login = true
tls_skip_verify_insecure = true
; Roles from Client roles in Keycloak
role_attribute_path = contains(resource_access.grafana.roles[*], 'Admin') && 'Admin' || contains(resource_access.grafana.roles[*], 'Editor') && 'Editor' || 'Viewer'
I can be redirected to Keycloak login page, but after login grafana has this error:
t=2021-10-15T11:48:58+0000 lvl=eror msg=login.OAuthLogin(NewTransportWithCode) logger=context userId=0 orgId=0 uname= error="oauth2: cannot fetch token: 400 Bad Request\nResponse: {\"error\":\"invalid_grant\",\"error_description\":\"Code not valid\"}"
t=2021-10-15T11:48:58+0000 lvl=eror msg="Request Completed" logger=context userId=0 orgId=0 uname= method=GET path=/login/generic_oauth status=500 remote_addr=172.18.0.1 time_ms=647 size=733 referer=
Keycloak configuration for grafana client:
What happens? What I am missing from configuration?
EDIT:
Grafana URL: http://localhost:13000
Keycloak logs:
16:38:09,650 ERROR [org.keycloak.services.error.KeycloakErrorHandler] (default task-4) Uncaught server error: java.lang.RuntimeException: cannot map type for token claim
...
...
16:38:09,942 WARN [org.keycloak.protocol.oidc.utils.OAuth2CodeParser] (default task-4) Code 'f72beb89-f814-4993-aa8f-e8debfea41ae' already used for userSession '6de1f56b-9c61-42ae-86bd-66d0ac7ad751' and client '36930d87-854f-414a-8177-c8237edf805c'.
16:38:09,944 WARN [org.keycloak.events] (default task-4) type=CODE_TO_TOKEN_ERROR, realmId=mcs, clientId=grafana, userId=null, ipAddress=172.16.1.1, error=invalid_code, grant_type=authorization_code, code_id=6de1f56b-9c61-42ae-86bd-66d0ac7ad751, client_auth_method=client-secret

Running terraform apply gives Unauthorized for Kubernetes provider deploying k8s resources in Azure Cloud

I'm deploying an AKS k8s cluster with terraform.
The cluster has rbac enabled with azure active directory.
The cluster creation goes fine and after that terraform tries to perfom some taks on the cluster like creating k8s-roles storage classes...., and fails there with an Unauthorized error message, like this :
module.k8s_cluster.module.infra.kubernetes_storage_class.managed-premium-retain: Creating...
module.k8s_cluster.module.infra.kubernetes_cluster_role.containerlogs: Creating...
module.k8s_cluster.module.infra.kubernetes_namespace.add_pod_identity: Creating...
module.k8s_cluster.module.infra.kubernetes_storage_class.managed-standard-retain: Creating...
module.k8s_cluster.module.infra.kubernetes_storage_class.managed-premium-delete: Creating...
module.k8s_cluster.module.appgw.kubernetes_namespace.agic[0]: Creating...
module.k8s_cluster.module.infra.kubernetes_storage_class.managed-standard-delete: Creating...
Error: Unauthorized
on .terraform/modules/k8s_cluster/modules/infra/k8s-roles.tf line 1, in resource "kubernetes_cluster_role" "containerlogs":
1: resource "kubernetes_cluster_role" "containerlogs" {
Error: Unauthorized
on .terraform/modules/k8s_cluster/modules/infra/k8s-storages-classes.tf line 1, in resource "kubernetes_storage_class" "managed-standard-retain":
1: resource "kubernetes_storage_class" "managed-standard-retain" {
Error: Unauthorized
on .terraform/modules/k8s_cluster/modules/infra/k8s-storages-classes.tf line 14, in resource "kubernetes_storage_class" "managed-standard-delete":
14: resource "kubernetes_storage_class" "managed-standard-delete" {
Error: Unauthorized
on .terraform/modules/k8s_cluster/modules/infra/k8s-storages-classes.tf line 27, in resource "kubernetes_storage_class" "managed-premium-retain":
27: resource "kubernetes_storage_class" "managed-premium-retain" {
Error: Unauthorized
on .terraform/modules/k8s_cluster/modules/infra/k8s-storages-classes.tf line 40, in resource "kubernetes_storage_class" "managed-premium-delete":
40: resource "kubernetes_storage_class" "managed-premium-delete" {
Error: Unauthorized
on .terraform/modules/k8s_cluster/modules/infra/r-aad-pod-identity.tf line 5, in resource "kubernetes_namespace" "add_pod_identity":
5: resource "kubernetes_namespace" "add_pod_identity" {
Error: Unauthorized
on .terraform/modules/k8s_cluster/modules/tools/agic/helm-agic.tf line 1, in resource "kubernetes_namespace" "agic":
1: resource "kubernetes_namespace" "agic" {
As you can see these are not azure errors, but kubernetes
It seems like I don't have rights to perform the above resources creation task on the newly created cluster.
What and where to do in order to grant my user account permissions for these terraform task?
An answer could be to change kubernetes provider configuration from
provider "kubernetes" {
load_config_file = "false"
host = azurerm_kubernetes_cluster.main.kube_config.0.host
username = azurerm_kubernetes_cluster.main.kube_config.0.username
password = azurerm_kubernetes_cluster.main.kube_config.0.password
client_certificate = "${base64decode(azurerm_kubernetes_cluster.main.kube_config.0.client_certificate)}"
client_key = "${base64decode(azurerm_kubernetes_cluster.main.kube_config.0.client_key)}"
cluster_ca_certificate = "${base64decode(azurerm_kubernetes_cluster.main.kube_config.0.cluster_ca_certificate)}"
}
to
provider "kubernetes" {
load_config_file = "false"
host = azurerm_kubernetes_cluster.main.kube_admin_config.0.host
username = azurerm_kubernetes_cluster.main.kube_admin_config.0.username
password = azurerm_kubernetes_cluster.main.kube_admin_config.0.password
client_certificate = "${base64decode(azurerm_kubernetes_cluster.main.kube_admin_config.0.client_certificate)}"
client_key = "${base64decode(azurerm_kubernetes_cluster.main.kube_admin_config.0.client_key)}"
cluster_ca_certificate = "${base64decode(azurerm_kubernetes_cluster.main.kube_admin_config.0.cluster_ca_certificate)}"
}
Note that if you disable the local account usage on your cluster this solution doesn't work.

VAULT_CLIENT_TOKEN keeps expiring every 24h

Environment:
Vault + Consul, all latest. Integrating Concourse (3.14.0) with Vault. All tokens and keys are throw-away. This is just a test cluster.
Problem:
No matter what I do, I get 768h as the token_duration value. Also, overnight my approle token keeps expiring no matter what I do. I have to regenerate token and pass it to Concourse and restart the service. I want this token not to expire.
[root#k1 etc]# vault write auth/approle/login role_id="34b73748-7e77-f6ec-c5fd-90c24a5a98f3" secret_id="80cc55f1-bb8b-e96c-78b0-fe61b243832d" duration=0
Key Value
--- -----
token 9a6900b7-062d-753f-131c-a2ac7eb040f1
token_accessor 171aeb1c-d2ce-0261-e20f-8ed6950d1d2a
token_duration 768h
token_renewable true
token_policies ["concourse" "default"]
identity_policies []
policies ["concourse" "default"]
token_meta_role_name concourse
[root#k1 etc]#
So, I use token - 9a6900b7-062d-753f-131c-a2ac7eb040f1 for my Concourse to access secrets and all is good, until 24h later. It gets expired.
I set duration to 0, but It didn't help.
$ vault write auth/approle/role/concourse secret_id_ttl=0 period=0 policies=concourse secret_id_num_uses=0 token_num_uses=0
My modified vaultconfig.hcl looks like this:
storage "consul" {
address = "127.0.0.1:8500"
path = "vault/"
token = "95FBC040-C484-4D16-B489-AA732DB6ADF1"
#token = "0b4bc7c7-7eb0-4060-4811-5f9a7185aa6f"
}
listener "tcp" {
address = "0.0.0.0:8200"
cluster_address = "0.0.0.0:8201"
tls_min_version = "tls10"
tls_disable = 1
}
cluster_addr = "http://192.168.163.132:8201"
api_addr = "http://192.168.163.132:8200"
disable_mlock = true
disable_cache = true
ui = true
default_lease_ttl = 0
cluster_name = "testcluster"
raw_storage_endpoint = true
My Concourse policy is vanilla:
[root#k1 etc]# vault policy read concourse
path "concourse/*" {
policy = "read"
capabilities = ["read", "list"]
}
[root#k1 etc]#
Look up token - 9a6900b7-062d-753f-131c-a2ac7eb040f1
[root#k1 etc]# vault token lookup 9a6900b7-062d-753f-131c-a2ac7eb040f1
Key Value
--- -----
accessor 171aeb1c-d2ce-0261-e20f-8ed6950d1d2a
creation_time 1532521379
creation_ttl 2764800
display_name approle
entity_id 11a0d4ac-10aa-0d62-2385-9e8071fc4185
expire_time 2018-08-26T07:22:59.764692652-05:00
explicit_max_ttl 0
id 9a6900b7-062d-753f-131c-a2ac7eb040f1
issue_time 2018-07-25T07:22:59.238050234-05:00
last_renewal 2018-07-25T07:24:44.764692842-05:00
last_renewal_time 1532521484
meta map[role_name:concourse]
num_uses 0
orphan true
path auth/approle/login
policies [concourse default]
renewable true
ttl 2763645
[root#k1 etc]#
Any pointers, feedback is very appreciated.
Try setting the token_ttl and token_max_ttl parameters instead of the secret_id_ttl when creating the new AppRole.
You should also check your Vault default_lease_ttl and max_lease_ttl, they might be set to 24h

How can I use Route 53 as the DNS Challenge for Lets Encrypt in Traefik?

My local domain is home.turtlesystems.co.uk. I am using Traefik on a local Docker Swarm cluster within this domain.
As there is no direct Internet access to the cluster I cannot use the HTTPS challenge for Lets Encrypt so I am attempting to use Route53 as the DNS provider.
I have set up a Zone in Route53 for my home domain, which is a sub domain of turtlesystems.co.uk which I own.
My traefik.toml file looks like:
debug = true
defaultEntryPoints = ["http", "https"]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
# Enable ACME (Let's Encrypt) automate SSL
[acme]
email = "xxxxxxxxxxxxxxxxxxxx"
storage = "/etc/traefik/acme.json"
dnsProvider = "route53"
entryPoint = "https"
onDemand = true
onHostRule = true
acmeLogging = true
[[acme.domains]]
main = "home.turtlesystems.co.uk"
# Allow access to the Web UI
[web]
address = ":8080"
# Configure how docker will be run
[docker]
endpoint = "unix://var/run/docker.sock"
domain = "traefik"
watch = true
exposedbydefault = false
swarmmode = true
I have created a service for Portainer that has the following Traefik labels:
traefik.port=9000
traefik.docker.network=traefik-net
traefik.frontend.rule=Host:turtle-host-03.home.turtlesystems.co.uk;PathStripPrefix:/portainer
traefik.backend=portainer
traefik.enable=true
traefik.backend.loadbalancer=wrr
As I have acmeLogging enabled in the traefik.toml file I was hoping to get some more information about what is happening or not happening, but I only get the following INFO logs:
reverse_proxy.1.rqebssg613a8#turtle-host-03 | legolog: 2017/12/15 13:16:32 [INFO][home.turtlesystems.co.uk] AuthURL: https://acme-v01.api.letsencrypt.org/acme/authz/z52B_D2iHeITPqT_7K-Z-Y-ieir3VT4l1qGW6tShrd8
reverse_proxy.1.rqebssg613a8#turtle-host-03 | legolog: 2017/12/15 13:16:32 [INFO][turtle-host-03.home.turtlesystems.co.uk] AuthURL: https://acme-v01.api.letsencrypt.org/acme/authz/OxWRpDR3KZm4E0nGngVSRZgF3iE2nhQ3jlNaWtxbd08
reverse_proxy.1.rqebssg613a8#turtle-host-03 | legolog: 2017/12/15 13:16:32 [INFO][home.turtlesystems.co.uk] acme: Could not find solver for: tls-sni-01
reverse_proxy.1.rqebssg613a8#turtle-host-03 | legolog: 2017/12/15 13:16:32 [INFO][home.turtlesystems.co.uk] acme: Trying to solve DNS-01
reverse_proxy.1.rqebssg613a8#turtle-host-03 | time="2017-12-15T13:17:06Z" level=debug msg="Look for provided certificate to validate [turtle-host-03.home.turtlesystems.co.uk]..."
reverse_proxy.1.rqebssg613a8#turtle-host-03 | time="2017-12-15T13:17:06Z" level=debug msg="Look for provided certificate to validate [turtle-host-03.home.turtlesystems.co.uk]..."
reverse_proxy.1.rqebssg613a8#turtle-host-03 | time="2017-12-15T13:17:06Z" level=debug msg="No provided certificate found for domains [turtle-host-03.home.turtlesystems.co.uk], get ACME certificate."
reverse_proxy.1.rqebssg613a8#turtle-host-03 | time="2017-12-15T13:17:06Z" level=debug msg="Challenge GetCertificate turtle-host-03.home.turtlesystems.co.uk"
reverse_proxy.1.rqebssg613a8#turtle-host-03 | time="2017-12-15T13:17:06Z" level=debug msg="No provided certificate found for domains [turtle-host-03.home.turtlesystems.co.uk], get ACME certificate."
reverse_proxy.1.rqebssg613a8#turtle-host-03 | time="2017-12-15T13:17:06Z" level=debug msg="Challenge GetCertificate turtle-host-03.home.turtlesystems.co.uk"
reverse_proxy.1.rqebssg613a8#turtle-host-03 | time="2017-12-15T13:17:07Z" level=debug msg="Look for provided certificate to validate [turtle-host-03.home.turtlesystems.co.uk]..."
reverse_proxy.1.rqebssg613a8#turtle-host-03 | time="2017-12-15T13:17:07Z" level=debug msg="No provided certificate found for domains [turtle-host-03.home.turtlesystems.co.uk], get ACME certificate."
reverse_proxy.1.rqebssg613a8#turtle-host-03 | time="2017-12-15T13:17:07Z" level=debug msg="Challenge GetCertificate turtle-host-03.home.turtlesystems.co.uk"
reverse_proxy.1.rqebssg613a8#turtle-host-03 | legolog: 2017/12/15 13:17:10 [INFO][home.turtlesystems.co.uk] Checking DNS record propagation using [127.0.0.11:53]
As can be seen it is trying to use a DNS challenge, but I am not getting a certificate.
When I first set all this up it did all work, in fact I wrote a blog about it, but now it does not. When I look at my AWS account I can see that the AWS_ACCESS_KEY I have created for this purpose is being used, but nothing seems to be entered into the Zone.
I am passing AWS_ACCESS_KEY, AWS_SECRET_ACCESS_KEY and AWS_REGION into the Portainer service as environment variables.
Is there more logging I can turn on? Is there anyway to see logs in AWS for Route 53?
Update
After playing around with this I noticed that Traefik is trying to use 127.0.0.11:53 as the DNS server on which to try and check that the TXT record has been created.
I then added --dns and --dns-search to the Traefik service but this did not have any effect on the address that Trafik uses for DNS. Is there another option I can set in Traefik to force this?
Go to AWS, Create AIM custom policy
Paste the following JSON as the policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"route53:ChangeResourceRecordSets",
"route53:GetChange",
"route53:GetChangeDetails",
"route53:ListHostedZones"
],
"Resource": [
"*"
]
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:SetLoadBalancerListenerSSLCertificate"
],
"Resource": [
"*"
]
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"iam:ListServerCertificates",
"iam:GetServerCertificate",
"iam:UploadServerCertificate"
],
"Resource": [
"*"
]
}
]
}
name the policy "dnsChallenge" (or whatever you like)
Create new AIM user and attach above policy
Copy the new user's keys as you'll need to set them as environment variables
Go to AWS Route53 and look at the hosted zone.
You'll want 2 A records -- for yourdomain.com and *.yourdomain.com both pointing to the static IP of the host running traefik.
Copy down the Hosted zone ID for the domain you are wildcarding.
Define the following environment variables and make sure they are available when traefik starts.
export AWS_ACCESS_KEY_ID=*****************
export AWS_SECRET_ACCESS_KEY=**********************************
export AWS_HOSTED_ZONE_ID=*************
edit traefik.toml
[acme] # Automatically add Let's Encrypt Certificate.
email = "youremail#gmail.com"
storage= "acme.json" # Change to fully qualified and exposed path for docker
entryPoint = "https"
OnHostRule = false
acmelogging = true
# caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"
caServer = "https://acme-v02.api.letsencrypt.org/directory"
[acme.dnsChallenge]
provider = "route53"
delayBeforeCheck = 0
[[acme.domains]]
main = "*.yourdomain.com"
sans = ["yourdomain.com"]
From there its a good idea to run it from the command line and watch for the messages..
Adding on bhlowe's answer, I would use a more restricted IAM profile:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"route53:GetChange",
"route53:ListHostedZonesByName"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"route53:ChangeResourceRecordSets"
],
"Resource": [
"arn:aws:route53:::hostedzone/<INSERT_YOUR_HOSTED_ZONE_ID_HERE>"
]
}
]
}