I am writing a go-lang app and I need to:
Go to the sibling directory
tried with:
exec.Command("/bin/sh", "-c", "cd ..").Output()
And clone/update GitHub private repository:
git clone ....GitHub repository
I cannot accomplish neither of those tasks.
I tried GitHub/libgit2/git2go but on Ubuntu 16.04 libgit2 cannot understand https.
Thank you for any help.
Credits comes to #JimB :-)
func update_ghub(wg *sync.WaitGroup) {
var (
cmdOut []byte
err error
)
err = os.Chdir("/home/svitlana/go/src/realsiter/realster")
if err != nil {
log.Fatalln(err)
}
cmdName := "git"
cmdArgs := []string{"pull"}
if cmdOut, err = exec.Command(cmdName, cmdArgs...).Output(); err != nil {
fmt.Fprintln(os.Stderr, "There was an error running git rev-parse command: ", err)
os.Exit(1)
}
sha := string(cmdOut)
fmt.Println("Response:", sha)
wg.Done()
}
Related
I have a problem with mongodb website.
Below i send image with my issue. I built project in Golang where i want to connect with mongodb and if i tried I got error
server selection error: context deadline exceeded, current topology: { Type: Unknown, Servers: [{ Addr: localhost:27017, Type: Unknown, Last error: dial tcp [::1]:27017: connect: connection refused }, ] }
Here I have code where i trying to connect with mongo client
func Connect() *DB {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://localhost:27017"))
if err != nil {
log.Fatal(err)
}
err = client.Connect(ctx)
return &DB{
client: client,
}
}
Error on site
Not sure to which MongoDB you are trying to connect. In your code example, the DB is on your local machine.
Your code is missing the DB credentials.
You can try the following code to connect to the DB and verify your connectivity:
client, err := mongo.Connect(ctx, options.Client().ApplyURI("mongodb://admin:password#localhost:27017"))
if err != nil {
log.Fatal(err)
}
if err = client.Ping(context.TODO(), readpref.Primary()); err != nil {
log.Fatal(err)
}
log.Println("Connected to MongoDB")
I'm building a CLI application that would allow me to run an arbitrary command in my shell against any kube cluster in my kubeconfig that matches a given regex. I want to use the official client-go package to accomplish this, but for some reason, switching kube contexts is less than intuitive. So I'm starting by modifying the example out-of-cluster program in the repo, and I'm struggling with just switching the context to the one I specify. Here is the code I started with, which gets the number of pods in the cluster loaded in the kubeconfig:
package main
import (
"context"
"flag"
"fmt"
"path/filepath"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/client-go/util/homedir"
)
func main() {
var kubeconfig *string
if home := homedir.HomeDir(); home != "" {
kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
} else {
kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
}
flag.Parse()
// use the current context in kubeconfig
config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
if err != nil {
panic(err.Error())
}
// create the clientset
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
panic(err.Error())
}
pods, err := clientset.CoreV1().Pods("").List(context.TODO(), metav1.ListOptions{})
if err != nil {
panic(err.Error())
}
fmt.Printf("There are %d pods in the test cluster\n", len(pods.Items))
}
Unfortunately, I cannot figure out how to load a specific cluster with a name as defined in my kubeconfig. I would love to have a sort of SwitchContext("cluster-name") function, but the number of Configs, ClientConfigs, RawConfigs, and restclient.Configs are confusing me. Any help would be appreciated!
System: Ubuntu 22.04, Intel, kube server version 1.23.8-gke.1900, client version 1.25.3
You can override the current context via NewNonInteractiveDeferredLoadingClientConfig method from clientcmd package.
package main
import (
"context"
"flag"
"fmt"
"k8s.io/client-go/rest"
"path/filepath"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/client-go/util/homedir"
)
func main() {
var kubeconfig *string
if home := homedir.HomeDir(); home != "" {
kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
} else {
kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
}
flag.Parse()
// use the current context in kubeconfig
config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
if err != nil {
panic(err.Error())
}
// using `contextName` context in kubeConfig
contextName := "gce"
config, err = buildConfigWithContextFromFlags(contextName, *kubeconfig)
if err != nil {
panic(err)
}
// create the clientset
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
panic(err.Error())
}
pods, err := clientset.CoreV1().Pods("").List(context.TODO(), metav1.ListOptions{})
if err != nil {
panic(err.Error())
}
fmt.Printf("There are %d pods in the test cluster\n", len(pods.Items))
}
func buildConfigWithContextFromFlags(context string, kubeconfigPath string) (*rest.Config, error) {
return clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
&clientcmd.ClientConfigLoadingRules{ExplicitPath: kubeconfigPath},
&clientcmd.ConfigOverrides{
CurrentContext: context,
}).ClientConfig()
}
I am writing integration testing for my Go program. When I try to initiate my MongoDB (with Docker) running in replica set, it raise error:
server selection error: server selection timeout, current topology:
{
Type: ReplicaSetNoPrimary,
Servers: [
{
Addr: b3ec1125321a:27017,
Type: Unknown,
Last error: connection() error occurred during connection handshake: dial tcp: lookup b3ec1125321a on 10.225.109.87:53: no such host
},
]
}
So I try to create a minimal runnable code as below with comment:
package main
import (
"bytes"
"context"
"errors"
"fmt"
"net"
"os/exec"
"time"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
// getDockerPath returns the path for docker.
func getDockerPath() (string, error) {
return exec.LookPath("docker")
}
// getPort returns a useable port. (should use it as soon as you can)
func getPort() (int, error) {
l, err := net.Listen("tcp", ":0")
if err != nil {
return 0, fmt.Errorf("listen to port 0: %w", err)
}
defer l.Close()
return l.Addr().(*net.TCPAddr).Port, nil
}
// runMongo will run a MongoDB container in docker.
func runMongo(ctx context.Context) error {
dockerPath, err := getDockerPath()
if err != nil {
return fmt.Errorf("get docker path: %w", err)
}
mongoPort, err := getPort()
if err != nil {
return fmt.Errorf("get mongo port: %w", err)
}
mongoURL := fmt.Sprintf("mongodb://127.0.0.1:%d", mongoPort)
fmt.Printf("Starting a mongoDB listening on %s\n\n", mongoURL)
// Run MongoDB - build command and run.
args := []string{
"run", "-d", "-p", fmt.Sprintf("%d:27017", mongoPort),
"mongo",
"mongod", "--replSet=rs0",
}
startCmd := exec.CommandContext(ctx, dockerPath, args...)
stdout := &bytes.Buffer{}
startCmd.Stdout = stdout
if err := startCmd.Run(); err != nil {
return fmt.Errorf("run MongoDB container by docker: %w", err)
}
// Get the Docker container name.
containerName, err := stdout.ReadString('\n')
for err != nil {
containerName, err = stdout.ReadString('\n')
}
containerName = containerName[:6]
// **********************************************************************
// NOTICE: When I remove the code below to initiate the MongoDB's Replica
// Set. And remove the argument `--replSet rs0`, it works.
// **********************************************************************
//
// Initiate Mongo's Replica Set.
args = []string{
"exec", containerName,
"mongo", "--eval", "rs.initiate();",
}
err = errors.New("in loop")
for err != nil {
<-time.After(1 * time.Second)
initCmd := exec.CommandContext(ctx, dockerPath, args...)
fmt.Println(initCmd)
err = initCmd.Run()
if err != nil {
fmt.Println(err)
}
}
// Init MongoDB before use it.
client, err := mongo.Connect(ctx, options.Client().ApplyURI(mongoURL))
if err != nil {
return fmt.Errorf("cannot connect to MongoDB: %w", err)
}
indexName := "Namespace's name's unique index"
True := true
index, err := client.Database("tmp").Collection("namespace").Indexes().CreateOne(ctx, mongo.IndexModel{
Keys: bson.D{{Key: "name", Value: 1}},
Options: &options.IndexOptions{
Name: &indexName,
Unique: &True,
},
})
if err != nil {
return fmt.Errorf("create index for namespace: %w", err)
}
fmt.Println(index)
return nil
}
func main() {
err := runMongo(context.Background())
if err != nil {
fmt.Printf("Fail to run MongoDB: %v\n", err)
}
}
When I try to run, it throw an error:
$ go run .
Starting a mongoDB listening on mongodb://127.0.0.1:29597
/usr/bin/docker exec b3ec11 mongo --eval rs.initiate();
Fail to run MongoDB: create index for namespace: server selection error: server selection timeout, current topology: { Type: ReplicaSetNoPrimary, Servers: [{ Addr: b3ec1125321a:27017, Type: Unknown, Last error: connection() error occurred during connection handshake: dial tcp: lookup b3ec1125321a on 10.225.109.87:53: no such host }, ] }
I try to search on Google. But still cannot find a solution.
UPDATE: I can use mongosh to do it without any error:
$ mongosh mongodb://127.0.0.1:29597
> use tmp
> ns = db.getCollection("namespace")
> ns.createIndex({name: 1}, {unique: true, name: "Namespace's name's unique index"})
And this really worry me a lot - I cannot get why I can use mongosh but not the go-driver of MongoDB.
It tells me that now it is ReplicaSetNoPrimary. But I try to use mongosh to connect it and run rs.isMaster() - it tells me that the only running MongoDB node IS the master node.
I try to let the MongoDB use the network host but not the bridge, and then it works:
--- another.go 2022-08-28 17:14:58.536381918 +0800
+++ main.go 2022-08-28 17:16:06.193467707 +0800
## -40,13 +40,13 ##
return fmt.Errorf("get mongo port: %w", err)
}
mongoURL := fmt.Sprintf("mongodb://127.0.0.1:%d", mongoPort)
- fmt.Printf("Starting a mongoDB listening on %s\n\n", mongoURL)
+ fmt.Printf("Starting a mongoDB listening on %s [on host network]\n\n", mongoURL)
// Run MongoDB - build command and run.
args := []string{
- "run", "-d", "-p", fmt.Sprintf("%d:27017", mongoPort),
+ "run", "-d", "--net=host",
"mongo",
- "mongod", "--replSet=rs0",
+ "mongod", "--replSet=rs0", fmt.Sprintf("--port=%d", mongoPort),
}
startCmd := exec.CommandContext(ctx, dockerPath, args...)
stdout := &bytes.Buffer{}
## -70,7 +70,7 ##
// Initiate Mongo's Replica Set.
args = []string{
"exec", containerName,
- "mongo", "--eval", "rs.initiate();",
+ "mongo", "--eval", "rs.initiate();", mongoURL,
}
err = errors.New("in loop")
for err != nil {
But I still do not know why I cannot let it work in bridge network. [Waiting for another answer still...]
I'm using mongodb community version 4.2.13 and go driver version 1.5.
My go application is running on the same host as db, but getting the following error when trying to make a connection:
connection() error occured during connection handshake: auth error:
sasl conversation error: unable to authenticate using mechanism
"SCRAM-SHA-256": (AuthenticationFailed) Authentication failed.
Here is how I created the admin account:
use admin
db.createUser({
user: "admin1",
pwd: "passwd12#$",
roles: ["root"],
mechanisms: ["SCRAM-SHA-256"]
})
db.system.users.update(
{ _id: "admin.admin1", "db": "admin" },
{
$addToSet: {
authenticationRestrictions: { clientSource: ["127.0.0.1"] }
}
}
)
Go app code snippet
package main
import (
"context"
"fmt"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
func main() {
uri := fmt.Sprintf(
"mongodb://%s:%s#%s:%d/admin?authSource=admin&authMechanism=SCRAM-SHA-256",
"admin1",
"passwd12#$",
"127.0.0.1",
27017,
)
// Prints "mongodb://admin1:passwd12#$#127.0.0.1:27017/admin?authSource=admin&authMechanism=SCRAM-SHA-256"
fmt.Println(uri)
ctx, cancel := context.WithTimeout(context.Background(), 10 * time.Second)
defer cancel()
client, err := mongo.Connect(
ctx,
options.Client().ApplyURI(uri),
)
if err != nil {
panic(err)
}
defer func() {
err = client.Disconnect(ctx)
if err != nil {
panic(err)
}
}()
err = client.Ping(ctx, nil)
if err != nil {
panic(err)
}
fmt.Println("pinged")
}
I tried the following, but all of them didn't work:
Encoding username and password using url.QueryEscape
Trying "localhost" instead of "127.0.0.1"
Removing "authMechanism=SCRAM-SHA-256" in uri
As a side note, connecting to the Mongo shell with the exact same uri, and that worked.
Add ssl=false to your uri. Worked for me
Based on MongoDB documentation for the authentication process, there is a parameter to identify which database is used for authentication besides the target database on the URI.
While in mongoshell you can use this line
mongo "mongodb://Admin:${DBPASSWORD}#<host>:<port>/admin?authSource=admin"
I used that information to add ?authSource=admin to my CONNECTION_URL
CONNECTION_URL=mongodb://root:example#mongo:27017/my_database?retryWrites=true&w=majority&authSource=admin
That worked for me. Hope it does for you too.
For detailed information please review https://www.mongodb.com/features/mongodb-authentication
You could try using 'options.Credential' to pass the authentication settings.
Seems like a cleaner way than formatting an URI that needs to be parsed later on.
https://docs.mongodb.com/drivers/go/current/fundamentals/auth/
clientOpts := options.Client().SetHosts(
[]string{"localhost:27017"},
).SetAuth(
options.Credential{
AuthSource: "<authenticationDb>",
AuthMechanism: "SCRAM-SHA-256",
Username: "<username>",
Password: "<password>",
}
)
client, err := mongo.Connect(context.TODO(), clientOpts)
I am creating a project in Go and I am using both "github.com/docker/docker/client" and "github.com/docker/docker/api/types", but when I try and create a container I get the following error:
ERROR: 2016/10/03 22:39:26 containers.go:84: error during connect: Post https://%2Fvar%2Frun%2Fdocker.sock/v1.23/containers/create: http: server gave HTTP response to HTTPS client
I can't understand why this is happening and it only happened after using the new golang docker engine(the old "github.com/docker/engine-api" is now deprecated).
The code isn't anything complicated, so I wonder if I am missing something:
resp, err := cli.Pcli.ContainerCreate(context.Background(), initConfig(), nil, nil, "")
if err != nil {
return err
}
And the initConfig that is called does the following:
func initConfig() (config *container.Config) {
mount := map[string]struct{}{"/root/host": {}}
return &container.Config{Image: "leadis_image", Volumes: mount, Cmd: strslice.StrSlice{"/root/server.py"}, AttachStdout: true}}
Also here is my dockerfile
FROM debian
MAINTAINER Leadis Journey
LABEL Description="This docker image is used to compile and execute user's program."
LABEL Version="0.1"
VOLUME /root/host/
RUN apt-get update && yes | apt-get upgrade
RUN yes | apt-get install gcc g++ python3 make
COPY container.py /root/server.py
EDIT
Just tried to test it with a simpler program
package main
import (
"fmt"
"os"
"io/ioutil"
"github.com/docker/docker/client"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/strslice"
"golang.org/x/net/context"
)
func initConfig() (config *container.Config) {
mount := map[string]struct{}{"/root/host": {}}
return &container.Config{Image: "leadis_image", Volumes: mount, Cmd: strslice.StrSlice{"/root/server.py"}, AttachStdout: true}
}
func main() {
client, _ := client.NewEnvClient()
cwd, _ := os.Getwd()
ctx, err := os.Open(cwd+"/Dockerfile.tar.gz")
if err != nil {
fmt.Println(err)
return
}
build, err := client.ImageBuild(context.Background(), ctx, types.ImageBuildOptions{Tags: []string{"leadis_image"}, Context: ctx, SuppressOutput: false})
if err != nil {
fmt.Println(err)
return
}
b, _ := ioutil.ReadAll(build.Body)
fmt.Println(string(b))
_, err = client.ContainerCreate(context.Background(), initConfig(), nil, nil, "")
if err != nil {
fmt.Println(err)
}
}
Same dockerfile, but I still get the same error:
error during connect: Post
https://%2Fvar%2Frun%2Fdocker.sock/v1.23/containers/create: http:
server gave HTTP response to HTTPS client
client.NewEnvClient()
Last time I tried, this API expects environment variables like DOCKER_HOST in a different syntax from than the normal docker client.
From the client.go:
// NewEnvClient initializes a new API client based on environment variables.
// Use DOCKER_HOST to set the url to the docker server.
// Use DOCKER_API_VERSION to set the version of the API to reach, leave empty for latest.
// Use DOCKER_CERT_PATH to load the TLS certificates from.
// Use DOCKER_TLS_VERIFY to enable or disable TLS verification, off by default.
To use this, you need DOCKER_HOST to be set/exported in one of these formats:
unix:///var/run/docker.sock
http://localhost:2375
https://localhost:2376