Go client SDK: check if a Deployment is fully ready - kubernetes

Say I have an in-memory Deployment object, what is the correct way of testing if it is fully ready? (not in the progress of rollout, upgrade or rollback).

I can't comment, so it will have to be an answer.
I don't think there is a right way of doing this, as it depends on a number of variables. Such as what languages you are proficient with etc.
Where I work, we run a kubectl get pods and grep the information that is relevant (in this case if the pod is available (ready) or not. This is all run through bash as part of a startup script:
function not_ready_count() {
kubectl ${1} get pods -o json | jq -r '.items[].status.conditions[].status' | grep False | wc -l | awk '{ print $1 }'
}
function not_running_count() {
kubectl ${1} get pods -o json | jq -r '.items[].status.phase' | grep -v Running | wc -l | awk '{ print $1 }'
}
function wait_until_running_and_ready() {
sleep 2
while [[ "$(not_running_count ${1})" != "0" ]]; do
echo "waiting for $(not_ready_count ${1}) pods to start"
sleep 3
done
while [[ "$(not_ready_count ${1})" != "0" ]]; do
echo "waiting for $(not_ready_count ${1}) status probes to pass"
sleep 3
done
sleep 2
}

Following is my implementation to check if a deployment is Ready using client-go:
package main
import (
"context"
"flag"
"fmt"
"path/filepath"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/client-go/util/homedir"
)
func main() {
var kubeconfig *string
if home := homedir.HomeDir(); home != "" {
kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
} else {
kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
}
flag.Parse()
config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
if err != nil {
panic(err)
}
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
panic(err)
}
deploymentsClient := clientset.AppsV1().Deployments("my-namespace")
deploy, err := deploymentsClient.Get(context.Background(), "my-deploy", metav1.GetOptions{})
if err != nil {
panic(err)
}
if deploy != nil &&
deploy.Spec.Replicas != nil &&
*deploy.Spec.Replicas == deploy.Status.ReadyReplicas {
fmt.Println("Deployment is READY")
} else {
fmt.Println("Deployment is NOT READY")
}
}

Related

How to switch kubernetes contexts dynamically with client-go?

I'm building a CLI application that would allow me to run an arbitrary command in my shell against any kube cluster in my kubeconfig that matches a given regex. I want to use the official client-go package to accomplish this, but for some reason, switching kube contexts is less than intuitive. So I'm starting by modifying the example out-of-cluster program in the repo, and I'm struggling with just switching the context to the one I specify. Here is the code I started with, which gets the number of pods in the cluster loaded in the kubeconfig:
package main
import (
"context"
"flag"
"fmt"
"path/filepath"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/client-go/util/homedir"
)
func main() {
var kubeconfig *string
if home := homedir.HomeDir(); home != "" {
kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
} else {
kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
}
flag.Parse()
// use the current context in kubeconfig
config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
if err != nil {
panic(err.Error())
}
// create the clientset
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
panic(err.Error())
}
pods, err := clientset.CoreV1().Pods("").List(context.TODO(), metav1.ListOptions{})
if err != nil {
panic(err.Error())
}
fmt.Printf("There are %d pods in the test cluster\n", len(pods.Items))
}
Unfortunately, I cannot figure out how to load a specific cluster with a name as defined in my kubeconfig. I would love to have a sort of SwitchContext("cluster-name") function, but the number of Configs, ClientConfigs, RawConfigs, and restclient.Configs are confusing me. Any help would be appreciated!
System: Ubuntu 22.04, Intel, kube server version 1.23.8-gke.1900, client version 1.25.3
You can override the current context via NewNonInteractiveDeferredLoadingClientConfig method from clientcmd package.
package main
import (
"context"
"flag"
"fmt"
"k8s.io/client-go/rest"
"path/filepath"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/client-go/util/homedir"
)
func main() {
var kubeconfig *string
if home := homedir.HomeDir(); home != "" {
kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
} else {
kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
}
flag.Parse()
// use the current context in kubeconfig
config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
if err != nil {
panic(err.Error())
}
// using `contextName` context in kubeConfig
contextName := "gce"
config, err = buildConfigWithContextFromFlags(contextName, *kubeconfig)
if err != nil {
panic(err)
}
// create the clientset
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
panic(err.Error())
}
pods, err := clientset.CoreV1().Pods("").List(context.TODO(), metav1.ListOptions{})
if err != nil {
panic(err.Error())
}
fmt.Printf("There are %d pods in the test cluster\n", len(pods.Items))
}
func buildConfigWithContextFromFlags(context string, kubeconfigPath string) (*rest.Config, error) {
return clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
&clientcmd.ClientConfigLoadingRules{ExplicitPath: kubeconfigPath},
&clientcmd.ConfigOverrides{
CurrentContext: context,
}).ClientConfig()
}

Does anyone know the way to do this on a client-go or the API resources that kubectl describe pod uses

I cannot find the appropriate method to do this.
Does anyone know the way to do this on a client-go or the API resources that kubectl describe pod uses?
Here is an example code of getting pod with client-go:
/*
A demonstration of get pod using client-go
Based on client-go examples: https://github.com/kubernetes/client-go/tree/master/examples
To demonstrate, run this file with `go run <filename> --help` to see usage
*/
package main
import (
"context"
"flag"
"fmt"
"os"
"path/filepath"
v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/client-go/util/homedir"
)
func main() {
podName := flag.String("pod-name", "", "name of the required pod")
namespaceName := flag.String("namespace", "", "namespace of the required pod")
var kubeconfig *string
if config, exist := os.LookupEnv("KUBECONFIG"); exist {
kubeconfig = &config
} else if home := homedir.HomeDir(); home != "" {
kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
} else {
kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
}
flag.Parse()
config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
if err != nil {
panic(err)
}
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
panic(err)
}
podClient := clientset.CoreV1().Pods(*namespaceName)
fmt.Println("Getting pod...")
result, err := podClient.Get(context.TODO(), *podName, v1.GetOptions{})
if err != nil {
panic(err)
}
// Example fields
fmt.Printf("%+v\n", result.Name)
fmt.Printf("%+v\n", result.Namespace)
fmt.Printf("%+v\n", result.Spec.ServiceAccountName)
}
you can see it on the code of the kubectl describe command

Using K8S API to access pod [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 1 year ago.
Improve this question
With kubectl we can run the following command
kubectl exec -ti POD_NAME -- pwd
Can I do that from API level? I checked the POD API and seems it is missing there https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/
What I am looking for, is a UI tool to view the files in POD without extra dependency
UPDATE:
I found the following code to exec command in pod
package main
import (
"bytes"
"context"
"flag"
"fmt"
"path/filepath"
corev1 "k8s.io/api/core/v1"
_ "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/client-go/tools/remotecommand"
"k8s.io/client-go/util/homedir"
//
// Uncomment to load all auth plugins
// _ "k8s.io/client-go/plugin/pkg/client/auth"
//
// Or uncomment to load specific auth plugins
// _ "k8s.io/client-go/plugin/pkg/client/auth/azure"
// _ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
// _ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
// _ "k8s.io/client-go/plugin/pkg/client/auth/openstack"
)
func main() {
var kubeconfig *string
if home := homedir.HomeDir(); home != "" {
kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
} else {
kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
}
flag.Parse()
// use the current context in kubeconfig
config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
if err != nil {
panic(err.Error())
}
// create the clientset
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
panic(err.Error())
}
namespace := "stage"
pods, err := clientset.CoreV1().Pods(namespace).List(context.TODO(), metav1.ListOptions{})
if err != nil {
panic(err.Error())
}
fmt.Printf("There are %d pods in the cluster\n", len(pods.Items))
podName := "ubs-job-qa-0"
containerName := "ubs-job"
// https://github.com/kubernetes/kubernetes/blob/release-1.22/test/e2e/framework/exec_util.go
// https://zhimin-wen.medium.com/programing-exec-into-a-pod-5f2a70bd93bb
req := clientset.CoreV1().
RESTClient().
Post().
Resource("pods").
Name(podName).
Namespace(namespace).
SubResource("exec").
Param("container", containerName)
scheme := runtime.NewScheme()
if err := corev1.AddToScheme(scheme); err != nil {
panic("Cannot add scheme")
}
parameterCodec := runtime.NewParameterCodec(scheme)
req.VersionedParams(&corev1.PodExecOptions{
Stdin: false,
Stdout: true,
Stderr: true,
TTY: true,
Container: podName,
Command: []string{"ls", "-la", "--time-style=iso", "."},
}, parameterCodec)
exec, err := remotecommand.NewSPDYExecutor(config, "POST", req.URL())
if err != nil {
panic(err)
}
var stdout, stderr bytes.Buffer
err = exec.Stream(remotecommand.StreamOptions{
Stdin: nil,
Stdout: &stdout,
Stderr: &stderr,
})
if err != nil {
panic(err)
}
text := string(stdout.Bytes())
fmt.Println(text)
}
In your case, use of kubectl is the same as calling the api-server; which in turn call the kubelet on the node and exec your command in the pod namespace.
You can experiment like this:
kubectl proxy --port=8080 &
curl "localhost:8080/api/v1/namespaces/<namespace>/pods/<pod>/exec?command=pwd&stdin=false"
To copy file you can use: kubectl cp --help
You can leverage tools like Kubernetes Dashboard as UI tool or if you wanna go enterprise level, try Rancher

Kubernetes: need notification if a new namespace is created

So Kubernetes events seems to have options for watch for all kinds of things like pod up/down creation/deletion etc.. - but I want to watch for a namespace creation/deletion itself - and I can't find an option to do that.
I want to know if someone created a namespace or deleted one. Is this possible?
Rgds,
Gopa.
So I got it working, pasting the entire code below for anyone's future reference
package main
import (
"os"
"os/exec"
"os/signal"
"strings"
"syscall"
"time"
"github.com/golang/glog"
v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/util/runtime"
"k8s.io/client-go/informers"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
"k8s.io/client-go/tools/cache"
"k8s.io/client-go/tools/clientcmd"
)
func newNamespace(obj interface{}) {
ns := obj.(v1.Object)
glog.Error("New Namespace ", ns.GetName())
}
func modNamespace(objOld interface{}, objNew interface{}) {
}
func delNamespace(obj interface{}) {
ns := obj.(v1.Object)
glog.Error("Del Namespace ", ns.GetName())
}
func watchNamespace(k8s *kubernetes.Clientset) {
// Add watcher for the Namespace.
factory := informers.NewSharedInformerFactory(k8s, 5*time.Second)
nsInformer := factory.Core().V1().Namespaces().Informer()
nsInformerChan := make(chan struct{})
//defer close(nsInformerChan)
defer runtime.HandleCrash()
// Namespace informer state change handler
nsInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{
// When a new namespace gets created
AddFunc: func(obj interface{}) {
newNamespace(obj)
},
// When a namespace gets updated
UpdateFunc: func(oldObj interface{}, newObj interface{}) {
modNamespace(oldObj, newObj)
},
// When a namespace gets deleted
DeleteFunc: func(obj interface{}) {
delNamespace(obj)
},
})
factory.Start(nsInformerChan)
//go nsInformer.GetController().Run(nsInformerChan)
go nsInformer.Run(nsInformerChan)
}
func main() {
kconfig := os.Getenv("KUBECONFIG")
glog.Error("KCONFIG", kconfig)
var config *rest.Config
var clientset *kubernetes.Clientset
var err error
for {
if config == nil {
config, err = clientcmd.BuildConfigFromFlags("", kconfig)
if err != nil {
glog.Error("Cant create kubernetes config")
time.Sleep(time.Second)
continue
}
}
// creates the clientset
clientset, err = kubernetes.NewForConfig(config)
if err != nil {
glog.Error("Cannot create kubernetes client")
time.Sleep(time.Second)
continue
}
break
}
watchNamespace(clientset)
glog.Error("Watch started")
term := make(chan os.Signal, 1)
signal.Notify(term, os.Interrupt)
signal.Notify(term, syscall.SIGTERM)
select {
case <-term:
}
}
Events are namespaced, they show the events happening in a particular namespace. However -A|--all-namespaces would show events from all the namespaces.
If you want to track the lifecycle(creation|deletion|etc) of the namespaces,then audit.logs are simplest option if not the only option.
Example: Creation of namespace called my-ns:
kubectl create ns my-ns
kubectl get events -n my-ns
No resources found in my-ns namespace.
kubectl describe ns my-ns
Name: my-ns
Labels: <none>
Annotations: <none>
Status: Active
No resource quota.
No LimitRange resource.
Now Here is the output of audit.log at metadata level, this would tell the following:
who created
what created
when created
and lot more.
Example output:
{
"kind":"Event",
"apiVersion":"audit.k8s.io/v1",
"level":"Metadata",
"auditID":"d28619be-0cb7-4d3e-b195-51fb93ae6de4",
"stage":"ResponseComplete",
"requestURI":"/api/v1/namespaces?fieldManager=kubectl-create",
"verb":"create", #<------operation type
"user":{
"username":"kubernetes-admin", #<--------who created
"groups":[
"system:masters",
"system:authenticated"
]
},
"sourceIPs":[
"1.2.3.4"
],
"userAgent":"kubectl/v1.20.0 (linux/amd64) kubernetes/af46c47",
"objectRef":{
"resource":"namespaces", #<---what created
"name":"my-ns", #<---name of resource
"apiVersion":"v1"
},
"responseStatus":{
"metadata":{
},
"code":201
},
"requestReceivedTimestamp":"2021-09-24T16:44:28.094213Z", #<---when created.
"stageTimestamp":"2021-09-24T16:44:28.270294Z",
"annotations":{
"authorization.k8s.io/decision":"allow",
"authorization.k8s.io/reason":""
}
}
$ kubectl get ns --watch-only
# run "kubectl create ns test" from another terminal
test Active 0s
# run "kubectl delete ns test"
test Terminating 23s
test Terminating 28s
test Terminating 28s

Error when creating container with golang docker engine

I am creating a project in Go and I am using both "github.com/docker/docker/client" and "github.com/docker/docker/api/types", but when I try and create a container I get the following error:
ERROR: 2016/10/03 22:39:26 containers.go:84: error during connect: Post https://%2Fvar%2Frun%2Fdocker.sock/v1.23/containers/create: http: server gave HTTP response to HTTPS client
I can't understand why this is happening and it only happened after using the new golang docker engine(the old "github.com/docker/engine-api" is now deprecated).
The code isn't anything complicated, so I wonder if I am missing something:
resp, err := cli.Pcli.ContainerCreate(context.Background(), initConfig(), nil, nil, "")
if err != nil {
return err
}
And the initConfig that is called does the following:
func initConfig() (config *container.Config) {
mount := map[string]struct{}{"/root/host": {}}
return &container.Config{Image: "leadis_image", Volumes: mount, Cmd: strslice.StrSlice{"/root/server.py"}, AttachStdout: true}}
Also here is my dockerfile
FROM debian
MAINTAINER Leadis Journey
LABEL Description="This docker image is used to compile and execute user's program."
LABEL Version="0.1"
VOLUME /root/host/
RUN apt-get update && yes | apt-get upgrade
RUN yes | apt-get install gcc g++ python3 make
COPY container.py /root/server.py
EDIT
Just tried to test it with a simpler program
package main
import (
"fmt"
"os"
"io/ioutil"
"github.com/docker/docker/client"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/strslice"
"golang.org/x/net/context"
)
func initConfig() (config *container.Config) {
mount := map[string]struct{}{"/root/host": {}}
return &container.Config{Image: "leadis_image", Volumes: mount, Cmd: strslice.StrSlice{"/root/server.py"}, AttachStdout: true}
}
func main() {
client, _ := client.NewEnvClient()
cwd, _ := os.Getwd()
ctx, err := os.Open(cwd+"/Dockerfile.tar.gz")
if err != nil {
fmt.Println(err)
return
}
build, err := client.ImageBuild(context.Background(), ctx, types.ImageBuildOptions{Tags: []string{"leadis_image"}, Context: ctx, SuppressOutput: false})
if err != nil {
fmt.Println(err)
return
}
b, _ := ioutil.ReadAll(build.Body)
fmt.Println(string(b))
_, err = client.ContainerCreate(context.Background(), initConfig(), nil, nil, "")
if err != nil {
fmt.Println(err)
}
}
Same dockerfile, but I still get the same error:
error during connect: Post
https://%2Fvar%2Frun%2Fdocker.sock/v1.23/containers/create: http:
server gave HTTP response to HTTPS client
client.NewEnvClient()
Last time I tried, this API expects environment variables like DOCKER_HOST in a different syntax from than the normal docker client.
From the client.go:
// NewEnvClient initializes a new API client based on environment variables.
// Use DOCKER_HOST to set the url to the docker server.
// Use DOCKER_API_VERSION to set the version of the API to reach, leave empty for latest.
// Use DOCKER_CERT_PATH to load the TLS certificates from.
// Use DOCKER_TLS_VERIFY to enable or disable TLS verification, off by default.
To use this, you need DOCKER_HOST to be set/exported in one of these formats:
unix:///var/run/docker.sock
http://localhost:2375
https://localhost:2376