Error when creating container with golang docker engine - sockets

I am creating a project in Go and I am using both "github.com/docker/docker/client" and "github.com/docker/docker/api/types", but when I try and create a container I get the following error:
ERROR: 2016/10/03 22:39:26 containers.go:84: error during connect: Post https://%2Fvar%2Frun%2Fdocker.sock/v1.23/containers/create: http: server gave HTTP response to HTTPS client
I can't understand why this is happening and it only happened after using the new golang docker engine(the old "github.com/docker/engine-api" is now deprecated).
The code isn't anything complicated, so I wonder if I am missing something:
resp, err := cli.Pcli.ContainerCreate(context.Background(), initConfig(), nil, nil, "")
if err != nil {
return err
}
And the initConfig that is called does the following:
func initConfig() (config *container.Config) {
mount := map[string]struct{}{"/root/host": {}}
return &container.Config{Image: "leadis_image", Volumes: mount, Cmd: strslice.StrSlice{"/root/server.py"}, AttachStdout: true}}
Also here is my dockerfile
FROM debian
MAINTAINER Leadis Journey
LABEL Description="This docker image is used to compile and execute user's program."
LABEL Version="0.1"
VOLUME /root/host/
RUN apt-get update && yes | apt-get upgrade
RUN yes | apt-get install gcc g++ python3 make
COPY container.py /root/server.py
EDIT
Just tried to test it with a simpler program
package main
import (
"fmt"
"os"
"io/ioutil"
"github.com/docker/docker/client"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/strslice"
"golang.org/x/net/context"
)
func initConfig() (config *container.Config) {
mount := map[string]struct{}{"/root/host": {}}
return &container.Config{Image: "leadis_image", Volumes: mount, Cmd: strslice.StrSlice{"/root/server.py"}, AttachStdout: true}
}
func main() {
client, _ := client.NewEnvClient()
cwd, _ := os.Getwd()
ctx, err := os.Open(cwd+"/Dockerfile.tar.gz")
if err != nil {
fmt.Println(err)
return
}
build, err := client.ImageBuild(context.Background(), ctx, types.ImageBuildOptions{Tags: []string{"leadis_image"}, Context: ctx, SuppressOutput: false})
if err != nil {
fmt.Println(err)
return
}
b, _ := ioutil.ReadAll(build.Body)
fmt.Println(string(b))
_, err = client.ContainerCreate(context.Background(), initConfig(), nil, nil, "")
if err != nil {
fmt.Println(err)
}
}
Same dockerfile, but I still get the same error:
error during connect: Post
https://%2Fvar%2Frun%2Fdocker.sock/v1.23/containers/create: http:
server gave HTTP response to HTTPS client

client.NewEnvClient()
Last time I tried, this API expects environment variables like DOCKER_HOST in a different syntax from than the normal docker client.
From the client.go:
// NewEnvClient initializes a new API client based on environment variables.
// Use DOCKER_HOST to set the url to the docker server.
// Use DOCKER_API_VERSION to set the version of the API to reach, leave empty for latest.
// Use DOCKER_CERT_PATH to load the TLS certificates from.
// Use DOCKER_TLS_VERIFY to enable or disable TLS verification, off by default.
To use this, you need DOCKER_HOST to be set/exported in one of these formats:
unix:///var/run/docker.sock
http://localhost:2375
https://localhost:2376

Related

Using K8S API to access pod [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 1 year ago.
Improve this question
With kubectl we can run the following command
kubectl exec -ti POD_NAME -- pwd
Can I do that from API level? I checked the POD API and seems it is missing there https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/
What I am looking for, is a UI tool to view the files in POD without extra dependency
UPDATE:
I found the following code to exec command in pod
package main
import (
"bytes"
"context"
"flag"
"fmt"
"path/filepath"
corev1 "k8s.io/api/core/v1"
_ "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/client-go/tools/remotecommand"
"k8s.io/client-go/util/homedir"
//
// Uncomment to load all auth plugins
// _ "k8s.io/client-go/plugin/pkg/client/auth"
//
// Or uncomment to load specific auth plugins
// _ "k8s.io/client-go/plugin/pkg/client/auth/azure"
// _ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
// _ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
// _ "k8s.io/client-go/plugin/pkg/client/auth/openstack"
)
func main() {
var kubeconfig *string
if home := homedir.HomeDir(); home != "" {
kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
} else {
kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
}
flag.Parse()
// use the current context in kubeconfig
config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
if err != nil {
panic(err.Error())
}
// create the clientset
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
panic(err.Error())
}
namespace := "stage"
pods, err := clientset.CoreV1().Pods(namespace).List(context.TODO(), metav1.ListOptions{})
if err != nil {
panic(err.Error())
}
fmt.Printf("There are %d pods in the cluster\n", len(pods.Items))
podName := "ubs-job-qa-0"
containerName := "ubs-job"
// https://github.com/kubernetes/kubernetes/blob/release-1.22/test/e2e/framework/exec_util.go
// https://zhimin-wen.medium.com/programing-exec-into-a-pod-5f2a70bd93bb
req := clientset.CoreV1().
RESTClient().
Post().
Resource("pods").
Name(podName).
Namespace(namespace).
SubResource("exec").
Param("container", containerName)
scheme := runtime.NewScheme()
if err := corev1.AddToScheme(scheme); err != nil {
panic("Cannot add scheme")
}
parameterCodec := runtime.NewParameterCodec(scheme)
req.VersionedParams(&corev1.PodExecOptions{
Stdin: false,
Stdout: true,
Stderr: true,
TTY: true,
Container: podName,
Command: []string{"ls", "-la", "--time-style=iso", "."},
}, parameterCodec)
exec, err := remotecommand.NewSPDYExecutor(config, "POST", req.URL())
if err != nil {
panic(err)
}
var stdout, stderr bytes.Buffer
err = exec.Stream(remotecommand.StreamOptions{
Stdin: nil,
Stdout: &stdout,
Stderr: &stderr,
})
if err != nil {
panic(err)
}
text := string(stdout.Bytes())
fmt.Println(text)
}
In your case, use of kubectl is the same as calling the api-server; which in turn call the kubelet on the node and exec your command in the pod namespace.
You can experiment like this:
kubectl proxy --port=8080 &
curl "localhost:8080/api/v1/namespaces/<namespace>/pods/<pod>/exec?command=pwd&stdin=false"
To copy file you can use: kubectl cp --help
You can leverage tools like Kubernetes Dashboard as UI tool or if you wanna go enterprise level, try Rancher

unable to authenticate with mongodb golang driver

I'm using mongodb community version 4.2.13 and go driver version 1.5.
My go application is running on the same host as db, but getting the following error when trying to make a connection:
connection() error occured during connection handshake: auth error:
sasl conversation error: unable to authenticate using mechanism
"SCRAM-SHA-256": (AuthenticationFailed) Authentication failed.
Here is how I created the admin account:
use admin
db.createUser({
user: "admin1",
pwd: "passwd12#$",
roles: ["root"],
mechanisms: ["SCRAM-SHA-256"]
})
db.system.users.update(
{ _id: "admin.admin1", "db": "admin" },
{
$addToSet: {
authenticationRestrictions: { clientSource: ["127.0.0.1"] }
}
}
)
Go app code snippet
package main
import (
"context"
"fmt"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
func main() {
uri := fmt.Sprintf(
"mongodb://%s:%s#%s:%d/admin?authSource=admin&authMechanism=SCRAM-SHA-256",
"admin1",
"passwd12#$",
"127.0.0.1",
27017,
)
// Prints "mongodb://admin1:passwd12#$#127.0.0.1:27017/admin?authSource=admin&authMechanism=SCRAM-SHA-256"
fmt.Println(uri)
ctx, cancel := context.WithTimeout(context.Background(), 10 * time.Second)
defer cancel()
client, err := mongo.Connect(
ctx,
options.Client().ApplyURI(uri),
)
if err != nil {
panic(err)
}
defer func() {
err = client.Disconnect(ctx)
if err != nil {
panic(err)
}
}()
err = client.Ping(ctx, nil)
if err != nil {
panic(err)
}
fmt.Println("pinged")
}
I tried the following, but all of them didn't work:
Encoding username and password using url.QueryEscape
Trying "localhost" instead of "127.0.0.1"
Removing "authMechanism=SCRAM-SHA-256" in uri
As a side note, connecting to the Mongo shell with the exact same uri, and that worked.
Add ssl=false to your uri. Worked for me
Based on MongoDB documentation for the authentication process, there is a parameter to identify which database is used for authentication besides the target database on the URI.
While in mongoshell you can use this line
mongo "mongodb://Admin:${DBPASSWORD}#<host>:<port>/admin?authSource=admin"
I used that information to add ?authSource=admin to my CONNECTION_URL
CONNECTION_URL=mongodb://root:example#mongo:27017/my_database?retryWrites=true&w=majority&authSource=admin
That worked for me. Hope it does for you too.
For detailed information please review https://www.mongodb.com/features/mongodb-authentication
You could try using 'options.Credential' to pass the authentication settings.
Seems like a cleaner way than formatting an URI that needs to be parsed later on.
https://docs.mongodb.com/drivers/go/current/fundamentals/auth/
clientOpts := options.Client().SetHosts(
[]string{"localhost:27017"},
).SetAuth(
options.Credential{
AuthSource: "<authenticationDb>",
AuthMechanism: "SCRAM-SHA-256",
Username: "<username>",
Password: "<password>",
}
)
client, err := mongo.Connect(context.TODO(), clientOpts)

Access to metric-server from a pod

I have deployed the metric-server on my kubernetes cluster and it's just working fine as I run the following command:
kubectl get --raw /apis/metrics.k8s.io/v1beta1/nodes
I want to access the metric-server from a pod. For that, I use the service/metric-server's IP and it is in the same namespace that the metric-server is. The way I'm trying to access to the metric is like this:
myurl := fmt.Sprintf("https://%s:%s/apis/metrics.k8s.io/v1beta1/nodes/", serviceHost, servicePort)
u, err := url.Parse(myurl)
if err != nil {
panic(err)
}
req, err := http.NewRequest(httpMethod, u.String(), nil)
if err != nil {
log.Printf("Cant sned req: %s", err)
}
caToken, err := ioutil.ReadFile("/var/run/secrets/kubernetes.io/serviceaccount/token")
if err != nil {
panic(err) // cannot find token file
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", string(caToken)))
caCertPool := x509.NewCertPool()
caCert, err := ioutil.ReadFile("/var/run/secrets/kubernetes.io/serviceaccount/ca.crt")
if err != nil {
panic(err)
}
caCertPool.AppendCertsFromPEM(caCert)
client := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
RootCAs: caCertPool,
},
},
}
resp, err := client.Do(req)
if err != nil {
log.Printf("sending helm deploy payload failed: %s", err.Error())
panic(err)
}
This is not working nad the logs result for the pod is:
Get https://METRIC-SERVER-SERVICE-IP/apis/metrics.k8s.io/v1beta1/nodes: x509: certificate is valid for 127.0.0.1, not METRIC-SERVER-SERVICE-IP
Is this the right way to access the metric-server from a pod?
what i did so i can access the metrics-server on a local deployment:
set proper rbac
inside the pod :
export CURL_CA_BUNDLE=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
curl -H "Authorization: Bearer $TOKEN" -k https://10.100.123.57/apis/metrics.k8s.io/v1beta1/nodes

Golang clone private github repository

I am writing a go-lang app and I need to:
Go to the sibling directory
tried with:
exec.Command("/bin/sh", "-c", "cd ..").Output()
And clone/update GitHub private repository:
git clone ....GitHub repository
I cannot accomplish neither of those tasks.
I tried GitHub/libgit2/git2go but on Ubuntu 16.04 libgit2 cannot understand https.
Thank you for any help.
Credits comes to #JimB :-)
func update_ghub(wg *sync.WaitGroup) {
var (
cmdOut []byte
err error
)
err = os.Chdir("/home/svitlana/go/src/realsiter/realster")
if err != nil {
log.Fatalln(err)
}
cmdName := "git"
cmdArgs := []string{"pull"}
if cmdOut, err = exec.Command(cmdName, cmdArgs...).Output(); err != nil {
fmt.Fprintln(os.Stderr, "There was an error running git rev-parse command: ", err)
os.Exit(1)
}
sha := string(cmdOut)
fmt.Println("Response:", sha)
wg.Done()
}

Connection refused with Go + Postgres on Heroku

I am trying to connect to Heroku's Postgres using a Go. All is working fine locally.
The error I am receiving on Heroku is dial tcp 127.0.0.1:5432: connection refused.
I've confirmed my ability to connect to the database via psql on heroku's command line, and have confirmed that the database url config is correct. The code is clear enough, so I wonder if there is a lower-level problem.
The code is straightforward enough:
import (
"database/sql"
"github.com/coopernurse/gorp"
_ "github.com/lib/pq"
"os"
)
func openDb() *sql.DB {
connection := os.Getenv("DATABASE_URL")
db, err := sql.Open("postgres", connection)
if err != nil {
log.Println(err)
}
return db
}
...and am importing github.com/lib/pq. Go version is 1.1.2.
Heroku + Go is pretty particular about the connection strings. The URL-style doesn't seem to allow specification of sslmode=require, which Heroku insists upon.
The modified version uses pq to parse the URL into a traditional Postgres connection string, and appends the parameter:
import (
"database/sql"
"github.com/lib/pq"
"os"
)
func openDb() *sql.DB {
url := os.Getenv("DATABASE_URL")
connection, _ := pq.ParseURL(url)
connection += " sslmode=require"
db, err := sql.Open("postgres", connection)
if err != nil {
log.Println(err)
}
return db
}