I am trying to run a spring batch with remote partitioning on K8s cluster using spring-cloud-deployer-kubernetes. Eventhough I have configured a service account and mentioned in my application properties the below way
spring.cloud.deployer.kubernetes.deployment-service-account-name=scdf-sa
Still the master task is unable to spawn worker pods and it seems it does not pick the property while launching task from spring cloud dataflow UI and throws this error in master pod:
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://<IP>/api/v1/namespaces/test/pods/batchsampleappworker-aeghj644g. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. pods "batchsampleappworker-j3ljqq3de9" is forbidden: User "system:serviceaccount:test:default" cannot get resource "pods" in API group "" in the namespace "test".
PS: I am using spring-cloud-deployer-kubernetes version of 2.5.0
Please some hints on how to correctly configure service account?
Thanks in advance!
As per the official documentation of spring cloud dataflow here, adding the below to SCDF server config map solved the issue for me.
data:
application.yaml: |-
spring:
cloud:
dataflow:
task:
platform:
kubernetes:
accounts:
default:
deploymentServiceAccountName: myserviceaccountname
Related
When I try to create a Jenkins X Kubernetes cluster with GKE using this command:
jx create cluster gke --skip-login
The following exeption is thrown at the end of installation:
error creating cluster configuring Jenkins: creating Jenkins API token: after 3 attempts, last error: creating Jenkins Auth configuration: secrets "jenkins" not found
During installation I select the default settings and provide my own github settings, including generated personal access token, but I don't think that the github token is the issue in this case (I'm pretty sure all my github settings are correct)
The problem has been solved by using --tekton flag:
jx create cluster gke --skip-login --tekton
I'm trying to use Pulumi to create a Deployment with a linked Service in a Kubesail cluster. The Deployment is created fine but when Pulumi tries to create the Service an error is returned:
kubernetes:core:Service (service):
error: Plan apply failed: resource service was not successfully created by the Kubernetes API server : Could not create watcher for Endpoint objects associated with Service "service": unknown
The Service is correctly created in Kubesail and the error seems to be glaringly obvious that it can't do Pulumi's neat monitoring but the unknown error isn't so neat!
What might be being denied on the Kubernetes cluster such that Pulumi can't do the monitoring that would be different between a Deployment and a Service? Is there a way to skip the watching that I missed in the docs to get me past this?
I dug a little into the Pulumi source code and found the resource kinds it uses to track and used kubectl auth can-i and low and behold watching an endpoint is currently denied but watching replicaSets and the service themselves is not.
I have a setup with scdf-server on kubernetes working fine, it deploys each task in an on-demand pod on the very same default namespace, the one that hosts the scdf-server pod.
Now, I need to deploy a pod in another namespace and I can't find the argument/property to use in the scdf server dashboard for the pod to be created in the given namespace. Does anybody know how to find that? I tried spring.cloud.deployer.kubernetes.namespace, deployer.kubernetes.namespace, spring.cloud.deployer.kubernetes.environmentVariables, deployer.<app>.kubernetes.namespace, spring.cloud.dataflow.task.platform.kubernetes.namespace, scheduler.kubernetes.environmentVariables SPRING_CLOUD_SCHEDULER_KUBERNETES_NAMESPACE... as both 'properties' and 'arguments' text boxes...
This seems like a duplicate thread that was posted in SCDF gitter channel. The properties were described and pointed out in the commentary - more details here.
I'm trying to deploy Spinnaker into an Openshift cluster(v3.10) using Halyard. Everything seems to deploy OK up until the deployment of spin-redis-bootstrap. The hal deploy apply command eventually times out, with the following error in the spin-redis-bootstrap pod logs:
Redis master data doesn't exist, data won't be persistent!
mkdir: cannot create directory '/redis-master-data': Permission denied
[7] 01 Oct 17:21:04.443 # Can't chdir to '/redis-master-data': No such file or directory
Seems like a permissions issue. This error does not occur when deploying directly to Kubernetes(v1.10).
Does halyard use a specific service account to deploy the Spinnaker services, that I would need to grant additional permissions to?
Any help would be appreciated.
I was able to spin Redis for Spinnaker by changing Docker image to registry.access.redhat.com/rhscl/redis-32-rhel7 in deployment config.
The reason it was failing due to more strictly permissions in OpenShift.
I've followed the instructions to create an EKS cluster in AWS using Terraform.
https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html
I've also copied the output for connecting to the cluster to ~/.kube/config-eks. I've verified this successfully works as I've been able to connect to the cluster and manually deploy containers. However, now i'm trying to use the Terraform Kubernetes provider to connect to the cluster but cannot seem to be able to configure the provider properly.
I've configured the provider to use my kubectl configuration but when attempting to push a simple configmap, i get an error stating the following:
configmaps is forbidden: User "system:anonymous" cannot create configmaps in the namespace "kube-system"
I know that the provider is picking up part of the configuration but I cannot seem to get it to authenticate. I suspect this is because EKS uses heptio for authentication and i'm not sure if the K8s Go client used by Terraform can support heptio. However, given that Terraform released their AWS EKS support when EKS went GA, I'd doubt that they wouldn't also update their Terraform provider to work with it.
Is it possible to even do this now? Are there alternatives?
Exec auth was added here: https://github.com/kubernetes/client-go/commit/19c591bac28a94ca793a2f18a0cf0f2e800fad04
This is what is utilized for custom authentication plugins and was published Feb 7th.
Right now, Terraform doesn't support the new exec-based authentication provider, but there is an issue open with a workaround: https://github.com/terraform-providers/terraform-provider-kubernetes/issues/161
That said, if I get some free time I will work on a PR.