I'm using mongo-go-driver, parallel connect several ports to check whether the port is listened by mongodb
go version 1.12.3
mongo-go-driver v1.0
type BaseServerStatus struct {
Host string `bson:"host"`
Version string `bson:"version"`
Process string `bson:"process"`
Pid int64 `bson:"pid"`
Uptime int64 `bson:"uptime"`
UptimeMillis int64 `bson:"uptimeMillis"`
UptimeEstimate int64 `bson:"uptimeEstimate"`
LocalTime time.Time `bson:"localTime"`
}
func GetBaseServerStatus(ip, port string) (srvStatus *BaseServerStatus, err error) {
opts := options.Client()
opts.SetDirect(true)
opts.SetServerSelectionTimeout(1 * time.Second)
opts.SetConnectTimeout(2 * time.Second)
opts.SetSocketTimeout(2 * time.Second)
opts.SetMaxConnIdleTime(1 * time.Second)
opts.SetMaxPoolSize(1)
url := fmt.Sprintf("mongodb://%s:%s/admin", ip, port)
opts.ApplyURI(url)
ctx, _ := context.WithTimeout(context.Background(), 2*time.Second)
conn, err := mongo.Connect(ctx, opts)
if err != nil {
fmt.Printf("new %s:%s mongo connection error: %v\n", ip, port, err)
return
}
defer conn.Disconnect(ctx)
err = conn.Ping(ctx, nil)
if err != nil {
fmt.Printf("ping %s:%s ping error: %v\n", ip, port, err)
return
}
sr := conn.Database("admin").RunCommand(ctx, bson.D{{"serverStatus", 1}})
if sr.Err() != nil {
fmt.Printf("get %s:%s server status error: %v\n", ip, port, sr.Err())
return
}
srvStatus = new(BaseServerStatus)
err = sr.Decode(srvStatus)
return
}
func main() {
var wg sync.WaitGroup
//ips := []string{"xxx.xxx.xxx.xxx:22"}
ips := []string{"xxx.xxx.xxx.xxx:22", "xxx.xxx.xxx.xxx:80", "xxx.xxx.xxx.xxx:7005", "xxx.xxx.xxx.xxx:7017", "xxx.xxx.xxx.xxx:7006", "xxx.xxx.xxx.xxx:7016", "xxx.xxx.xxx.xxx:7018", "xxx.xxx.xxx.xxx:7014", "xxx.xxx.xxx.xxx:199", "xxx.xxx.xxx.xxx:8182", "xxx.xxx.xxx.xxx:7015", "xxx.xxx.xxx.xxx:7022", "xxx.xxx.xxx.xxx:7013", "xxx.xxx.xxx.xxx:7020", "xxx.xxx.xxx.xxx:9009", "xxx.xxx.xxx.xxx:7004", "xxx.xxx.xxx.xxx:7008", "xxx.xxx.xxx.xxx:7002", "xxx.xxx.xxx.xxx:7021", "xxx.xxx.xxx.xxx:7007", "xxx.xxx.xxx.xxx:7024", "xxx.xxx.xxx.xxx:7010", "xxx.xxx.xxx.xxx:7011", "xxx.xxx.xxx.xxx:7003", "xxx.xxx.xxx.xxx:7012", "xxx.xxx.xxx.xxx:7009", "xxx.xxx.xxx.xxx:7019", "xxx.xxx.xxx.xxx:8001", "xxx.xxx.xxx.xxx:7023", "xxx.xxx.xxx.xxx:111", "xxx.xxx.xxx.xxx:7001", "xxx.xxx.xxx.xxx:8002", "xxx.xxx.xxx.xxx:19313", "xxx.xxx.xxx.xxx:15772", "xxx.xxx.xxx.xxx:19777", "xxx.xxx.xxx.xxx:15778", "xxx.xxx.xxx.xxx:15776"}
for _, ip := range ips {
wg.Add(1)
//time.Sleep(3 * time.Second)
go func(addr string) {
fmt.Printf("start to probe port %s\n", addr)
GetBaseServerStatus(strings.Split(addr, ":")[0], strings.Split(addr, ":")[1])
wg.Done()
}(ip)
}
wg.Wait()
fmt.Println("scan end")
time.Sleep(20 * time.Second)
}
however, run this code ,it consume 26GB memory
I use pprof to diagnose, see below
Showing top 10 nodes out of 15
flat flat% sum% cum cum%
26.29GB 96.86% 96.86% 26.29GB 96.86% go.mongodb.org/mongo-driver/x/network/connection.(*connection).ReadWireMessage
Related
I want to create a UDP server.
The server has several network interfaces : I want each interface to respond even if it is not in the same subnet as clients (clients send broadcast Ethernet packets).
How to do it ?
// The UDP server receives the client's request but fails to respond if it is not in the same subnet mask.
func udp_server(local_interface_ip net.IP){
p := make([]byte, 2048)
addr := net.UDPAddr{
Port: 1234,
IP: local_interface_ip,
}
ser, err := net.ListenUDP("udp", &addr)
if err != nil {
fmt.Printf("Some error %v\r\n", err)
return
}
for {
packetLen, remoteaddr, err := ser.ReadFromUDP(p)
if err != nil {
fmt.Printf("Some error %v", err)
continue
}
fmt.Printf("[RX %v] message(len : %d) : %s\r\n", remoteaddr, packetLen, p)
sendResponse(ser, remoteaddr)
}
}
func sendResponse(conn *net.UDPConn, addr *net.UDPAddr) {
data := []byte("1234")
_, err := conn.WriteToUDP(data, addr)
if err != nil {
fmt.Printf("[TX] Couldn't send response to %v (%v)\r\n", addr, err)
}
}
Then, for the server response, I want to put in the UDP data the MAC address of the server interface: How to do?
// This function get all MAC Address of the server but to know what is the good interface ?
func getMacAddr() ([]string, error) {
ifas, err := net.Interfaces()
if err != nil {
return nil, err
}
var as []string
for _, ifa := range ifas {
a := ifa.HardwareAddr.String()
if a != "" {
as = append(as, a)
}
}
return as, nil
}
It is a custom protocol based on UDP which is imposed to me which allows the clients to be able to discover IP address and MAC address of server interfaces.
I have mongo capped collection and a simple API, written on Go. I built and run it. When I try to sent Get request or simply go localhost:8000/logger in browser - my process closes. Debug shows this happens, while executing "find" in collection. It produces error "client is disconnected". Collection has 1 document, and debug shows it is connected with my helper.
Go version 1.13
My code:
func main() {
r := mux.NewRouter()
r.HandleFunc("/logger", getDocs).Methods("GET")
r.HandleFunc("/logger", createDoc).Methods("POST")
log.Fatal(http.ListenAndServe("localhost:8000", r))
}
func getDocs(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
var docs []models.Logger
//Connection mongoDB with helper class
collection := helper.ConnectDB()
cur, err := collection.Find(context.TODO(), bson.M{})
if err != nil {
helper.GetError(err, w)
return
}
defer cur.Close(context.TODO())
for cur.Next(context.TODO()) {
var doc models.Logger
err := cur.Decode(&doc)
if err != nil {
log.Fatal(err)
}
docs = append(docs, doc)
}
if err := cur.Err(); err != nil {
log.Fatal(err)
}
json.NewEncoder(w).Encode(docs)
}
func ConnectDB() *mongo.Collection {
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://127.0.0.1:27017"))
if err != nil {
log.Fatal(err)
}
fmt.Println("Connected to MongoDB!")
logCollection := client.Database("local").Collection("loggerCollection")
return logCollection
}
According to the documentation, the call to mongo.NewClient doesn't ensure that you can connect the Mongo server. You should first call mongo.Client.Ping() to verify if you can connect to the database or not.
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://127.0.0.1:27017"))
if err != nil {
log.Fatal(err)
}
if err := client.Ping(context.TODO(), readpref.Primary()); err != nil {
// Can't connect to Mongo server
log.Fatal(err)
}
There could be several reasons behind failing to connect, the most obvious one is incorrect setup of ports. Is your mongodb server up and listening on port 27017? Is there any change you're running mongodb with Docker and it's not forwarding to the correct port?
I faced similar issue , read #Jay answer it definitely helped , as I checked my MongoDB was running using "MongoDB Compass" , then I changed the location of my insert statement , previously I was calling before the call of "context.WithTimeout". Below is working code.
package main
import (
"context"
"log"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
type Book struct {
Name string `json:"name,omitempty"`
PublisherID string `json:"publisherid,omitempty"`
Cost string `json:"cost,omitempty"`
StartTime string `json:"starttime,omitempty"`
EndTime string `json:"endtime,omitempty"`
}
func main() {
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://localhost:27017"))
if err != nil {
log.Fatal(err)
}
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
err = client.Connect(ctx)
if err != nil {
log.Fatal(err)
}
defer client.Disconnect(ctx)
testCollection := client.Database("BooksCollection").Collection("BooksRead")
inserRes, err := testCollection.InsertOne(context.TODO(), Book{Name: "Harry Potter", PublisherID: "IBN123", Cost: "1232", StartTime: "2013-10-01T01:11:18.965Z", EndTime: "2013-10-01T01:11:18.965Z"})
log.Println("InsertResponse : ", inserRes)
log.Println("Error : ", err)
}
I can see document inserted in console as well as in "MongoDB Comapass."
In heiper function "ConnectDB" after "NewClient" I must use "client.Connect(context.TODO())"
before any other use of client
I need help to fix the error. I'm using IBM mongo services.
go version go1.13.6 darwin/amd64
mongo driver version 1.2.1
The connection is working, I can read and write but sometimes it returns
: command find requires authentication and command insert requires authentication
MONGO_DB_URI=mongodb://username:password:port,host/dbname?authSource=admin&replicaSet=replset&connect=direct&alias=default
Connect:
func ConnectDatabase() *mongo.Client {
clientOptions := options.Client().ApplyURI(os.Getenv("MONGO_DB_URI"))
ctx, _ := context.WithTimeout(context.Background(), 30*time.Second)
var err error
client, err = mongo.Connect(ctx, clientOptions)
if err != nil {
log.Fatal(err)
}
ctx, _ = context.WithTimeout(context.Background(), 10*time.Second)
err = client.Ping(ctx, nil)
if err != nil {
log.Fatal(err)
return nil
}
fmt.Println("Connected to MongoDB!")
return client
}
Read:
func FindAll(collectionName string, query bson.M) (*mongo.Cursor, error) {
collection := client.Database("dbname").Collection(collectionName)
singleResult, err := collection.Find(context.TODO(), query)
return singleResult, err
}
Read:
ctx, _ := context.WithTimeout(context.Background(), 20*time.Second)
cur, err := mongo.GetCollection("collection_name").Find(ctx, createQuery())
if err != nil {
log.Println(err.Error())
}
I'm using the same database and the same configurations at our another Python Project. No exceptions.
There is a difference between connecting to a DB and performing operations on the DB.
Mongo lets you connect without authentication because you have to be able to connect to be able to authenticate.
var cred options.Credential
cred.AuthSource = YourAuthSource
cred.Username = YourUserName
cred.Password = YourPassword
// set client options
clientOptions := options.Client().ApplyURI(os.Getenv("MONGO_DB_URI")).SetAuth(cred)
//... the rest of your code
Hope this helps.
I am trying to create a TCP server and client using Golang where I am able to set the Type of Service field in the IP header in order to prioritise different traffic flows.
The client and servers are able to communicate but I can not figure out how to set the ToS field.
I have tried using the ipv4 Golang package with the method described here: https://godoc.org/golang.org/x/net/ipv4#NewConn
A simplified server example:
func main () {
ln, err := net.Listen("tcp4", "192.168.0.20:1024")
if err != nil {
// error handling
}
defer ln.Close()
for {
c, err := ln.Accept()
if err != nil {
// error handling
}
go func(c net.Conn) {
defer c.Close()
if err := ipv4.NewConn(c).SetTOS(0x28); err != nil {
fmt.Println("Error: ", err.Error())
}
}(c)
}
And the corresponding client (also simplified)
func main () {
conn, err := net.Dial("tcp4", "192.168.0.20:1024")
if err != nil {
fmt.Println(err)
}
for {
writer := bufio.NewWriter(conn)
// Create "packet"
Data := make([]byte, 1200)
endLine := "\r\n"
//Set packetLength
length := strconv.FormatInt(int64(1200), 10)
copy(Data[0:], length)
//Set ID
idString := strconv.FormatInt(int64(1), 10)
if strings.Contains(idString, "\r") || strings.Contains(idString, "\n") || strings.Contains(idString, "\r\n") {
fmt.Println("This is gonna result in an error in the id string.")
}
idbuf := []byte(idString)
copy(Data[15:], idbuf)
//Set timestamp
timestamp0 := time.Now().UnixNano()
timestampString := strconv.FormatInt(timestamp0, 10)
if strings.Contains(timestampString, "\r") || strings.Contains(timestampString, "\n") || strings.Contains(timestampString, "\r\n") {
fmt.Println("This is gonna result in an error in the timestamp string.")
}
buf := []byte(timestampString)
copy(Data[50:], buf)
copy(Data[int(1200)-2:], endLine)
if len(Data) != int(1200) {
fmt.Println("This is also gonna be an error. Length is: ", len(Data))
}
//Send the data and flush the writer
writer.Write(Data)
writer.Flush()
}
//time.Sleep(1*time.Nanosecond)
}
I have also tried creating my own dialer with a control function that passes a syscall in order to set the socket like this:
dialer := &net.Dialer{
Timeout: 5 * time.Second,
Deadline: time.Time{},
LocalAddr: tcpAddr,
DualStack: false,
FallbackDelay: 0,
KeepAlive: 0,
Resolver: nil,
Control: highPrio,
}
func highPrio(network, address string, c syscall.RawConn) error {
return c.Control(func(fd uintptr) {
// set the socket options
err := syscall.SetsockoptInt(syscall.Handle(fd), syscall.IPPROTO_IP, syscall.IP_TOS, 128)
if err != nil {
log.Println("setsocketopt: ", err)
}
})
I am verifying that it does not work by inspecting the traffic with Wireshark and am using Windows 10 Pro as my OS.
I am try you ToS set method at Dial() with golang 1.15.5 and its worked:
dialer := net.Dialer{
Timeout: this.TcpWaitConnectTimeout,
}
dialer.Control = func(network, address string, c syscall.RawConn) error {
var err error
c.Control(func(fd uintptr) {
err = syscall.SetsockoptInt(int(fd), syscall.IPPROTO_IP, syscall.IP_TOS, 0x80)
})
return err
}
c, err := dialer.Dial("tcp", this.serverAddr)
tcpdump show me right ToS
I tried to connect to mongodb Atlas using golang drivers.
tlsConfig := &tls.Config{}
var mongoURI = "mongodb+srv://admin:password#prefix.mongodb.net:27017/dbname"
dialInfo, err := mgo.ParseURL(mongoURI)
if err != nil {
panic(err)
}
dialInfo.DialServer = func(addr *mgo.ServerAddr) (net.Conn, error) {
conn, err := tls.Dial("tcp", addr.String(), tlsConfig)
return conn, err
}
session, err := mgo.DialWithInfo(dialInfo)
if err != nil {
println("error")
log.Fatal(err)
}
_ = session
c := session.DB("Token").C("_Users")
user := &User{firstName: "username"}
err = c.Insert(user)
if err != nil {
println("error Again")
}
I am not getting an error not getting connected.
I am wondering what could be the reason.'
Any help is appreciated.
I tried to create DialInfo using below code
dialInfo := &mgo.DialInfo{
Addrs: []string{"prefix.mongodb.net:27017"},
Database: "dbname",
Mechanism: "SCRAM",
Timeout: 10 * time.Second,
Username: "admin",
Password: "passwrd",
}
Now I am getting no reachable servers
I could only see that the code started, then nothing
As you have figured out, this is because DialInfo by default has a zero timeout. The call will block forever waiting for a connection to be established. You can also specify a timeout with:
dialInfo.Timeout = time.Duration(30)
session, err := mgo.DialWithInfo(dialInfo)
Now I am getting no reachable servers
This is because globalsign/mgo does not currently support SRV connection string URI yet. See issues 112.
You can use the non-srv connection URI format (MongoDB v3.4), see a related question StackOverflow: 41173720.
You can use mongo-go-driver instead if you would like to connect using the SRV connection URI, for example:
mongoURI := "mongodb+srv://admin:password#prefix.mongodb.net/dbname?ssl=true&retryWrites=true"
client, err := mongo.NewClient(options.Client().ApplyURI(mongoURI))
if err != nil {
log.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
err = client.Connect(ctx)
defer client.Disconnect(ctx)
if err != nil {
log.Fatal(err)
}
database := client.Database("go")
collection := database.Collection("atlas")
The above example is compatible with the current version v1.0.0
For MongoDB Atlas
serverAPIOptions := options.ServerAPI(options.ServerAPIVersion1)
clientOptions := options.Client().
ApplyURI("mongodb://username:password#prefix0.mongodb.net:27017,prefix1.mongodb.net:27017,prefix2.mongodb.net:27017/?retryWrites=true&w=majority&replicaSet=atlas-zhqegh-shard-0&tls=true").
SetServerAPIOptions(serverAPIOptions)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
client, err := mongo.Connect(ctx, clientOptions)
if err != nil {
log.Fatal(err)
}
To clarify the MongoDB Atlas replicaSet and hosts you can for instance utilize MongoDB Compass: just connect to the cluster and you will see all that data.