Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 2 years ago.
Improve this question
I have been using MongoDB without issues in Python but I am in need to build a client in go now.
I have looked at the documentation and the examples work fine.
But when I try to use my own code, the code execute without errors, but when I inspect the database (via CLI) I see no database, no collection and no data.
I am sure I am doing something wrong, but I am unable to find it in this little testing code.
func main() {
if client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://localhost:27017")); err == nil {
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
defer client.Disconnect(ctx)
if err = client.Connect(ctx); err == nil {
os.Exit(0)
}
KeysCol := client.Database("yfroot").Collection("KeysCol")
mac, e1 := crypto.GenerateRandomBytes(16)
key, e2 := crypto.GenerateRandomBytes(16)
if e1 != nil || e2 != nil {
fmt.Println("failed crypto generate")
os.Exit(0)
}
testKey := dataformats.DeviceKey{
Mac: string(mac),
Key: key,
}
// ctx, _ = context.WithTimeout(context.Background(), time.Duration(10)*time.Second)
_, err := KeysCol.InsertOne(ctx, testKey)
if err != nil {
fmt.Println(err)
os.Exit(0)
}
} else {
fmt.Println(err)
}
}
Look at this part:
if err = client.Connect(ctx); err == nil {
os.Exit(0)
}
If you are able to connect, that us (err is nil), you are exiting.
You probably meant to do:
if err = client.Connect(ctx); err != nil {
fmt.Println(err)
os.Exit(0)
}
Related
Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 1 year ago.
Improve this question
I am facing the above issue with the code below
stmt, err2 := db.Prepare( "SELECT COUNT(*) FROM xyz WHERE product_id=? and chart_number=?")
rows, err2 := stmt.Query( bidStatusReqVal.ProductId,bidStatusReqVal.ChartNumber).Scan(&count)
Query(...).Scan(...) is not valid because Query returns two values and chaining of calls requires that the previous call returns only one value. Call Scan on the returned rows, or use QueryRow(...).Scan(...) with only err as the return destination.
rows, err := stmt.Query(bidStatusReqVal.ProductId, bidStatusReqVal.ChartNumber)
if err != nil {
return err
}
defer rows.Close()
for rows.Next() {
if err := rows.Scan(&count); err != nil {
return err
}
}
if err := rows.Err(); err != nil {
return err
}
// ...
In cases where the query returns only a single row, e.g. SELECT ... LIMIT 1, or SELECT COUNT(*) ... like in your case, it is much more convenient to use QueryRow.
err := stmt.QueryRow(bidStatusReqVal.ProductId, bidStatusReqVal.ChartNumber).Scan(&count)
if err != nil {
return err
}
// ...
I have mongo capped collection and a simple API, written on Go. I built and run it. When I try to sent Get request or simply go localhost:8000/logger in browser - my process closes. Debug shows this happens, while executing "find" in collection. It produces error "client is disconnected". Collection has 1 document, and debug shows it is connected with my helper.
Go version 1.13
My code:
func main() {
r := mux.NewRouter()
r.HandleFunc("/logger", getDocs).Methods("GET")
r.HandleFunc("/logger", createDoc).Methods("POST")
log.Fatal(http.ListenAndServe("localhost:8000", r))
}
func getDocs(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
var docs []models.Logger
//Connection mongoDB with helper class
collection := helper.ConnectDB()
cur, err := collection.Find(context.TODO(), bson.M{})
if err != nil {
helper.GetError(err, w)
return
}
defer cur.Close(context.TODO())
for cur.Next(context.TODO()) {
var doc models.Logger
err := cur.Decode(&doc)
if err != nil {
log.Fatal(err)
}
docs = append(docs, doc)
}
if err := cur.Err(); err != nil {
log.Fatal(err)
}
json.NewEncoder(w).Encode(docs)
}
func ConnectDB() *mongo.Collection {
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://127.0.0.1:27017"))
if err != nil {
log.Fatal(err)
}
fmt.Println("Connected to MongoDB!")
logCollection := client.Database("local").Collection("loggerCollection")
return logCollection
}
According to the documentation, the call to mongo.NewClient doesn't ensure that you can connect the Mongo server. You should first call mongo.Client.Ping() to verify if you can connect to the database or not.
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://127.0.0.1:27017"))
if err != nil {
log.Fatal(err)
}
if err := client.Ping(context.TODO(), readpref.Primary()); err != nil {
// Can't connect to Mongo server
log.Fatal(err)
}
There could be several reasons behind failing to connect, the most obvious one is incorrect setup of ports. Is your mongodb server up and listening on port 27017? Is there any change you're running mongodb with Docker and it's not forwarding to the correct port?
I faced similar issue , read #Jay answer it definitely helped , as I checked my MongoDB was running using "MongoDB Compass" , then I changed the location of my insert statement , previously I was calling before the call of "context.WithTimeout". Below is working code.
package main
import (
"context"
"log"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
type Book struct {
Name string `json:"name,omitempty"`
PublisherID string `json:"publisherid,omitempty"`
Cost string `json:"cost,omitempty"`
StartTime string `json:"starttime,omitempty"`
EndTime string `json:"endtime,omitempty"`
}
func main() {
client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://localhost:27017"))
if err != nil {
log.Fatal(err)
}
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
err = client.Connect(ctx)
if err != nil {
log.Fatal(err)
}
defer client.Disconnect(ctx)
testCollection := client.Database("BooksCollection").Collection("BooksRead")
inserRes, err := testCollection.InsertOne(context.TODO(), Book{Name: "Harry Potter", PublisherID: "IBN123", Cost: "1232", StartTime: "2013-10-01T01:11:18.965Z", EndTime: "2013-10-01T01:11:18.965Z"})
log.Println("InsertResponse : ", inserRes)
log.Println("Error : ", err)
}
I can see document inserted in console as well as in "MongoDB Comapass."
In heiper function "ConnectDB" after "NewClient" I must use "client.Connect(context.TODO())"
before any other use of client
Talk is cheap, so here we go the simple code:
package main
import (
"fmt"
"time"
"net"
)
func main() {
addr := "127.0.0.1:8999"
// Server
go func() {
tcpaddr, err := net.ResolveTCPAddr("tcp4", addr)
if err != nil {
panic(err)
}
listen, err := net.ListenTCP("tcp", tcpaddr)
if err != nil {
panic(err)
}
for {
if conn, err := listen.Accept(); err != nil {
panic(err)
} else if conn != nil {
go func(conn net.Conn) {
buffer := make([]byte, 1024)
n, err := conn.Read(buffer)
if err != nil {
fmt.Println(err)
} else {
fmt.Println(">", string(buffer[0 : n]))
}
conn.Close()
}(conn)
}
}
}()
time.Sleep(time.Second)
// Client
if conn, err := net.Dial("tcp", addr); err == nil {
for i := 0; i < 2; i++ {
_, err := conn.Write([]byte("hello"))
if err != nil {
fmt.Println(err)
conn.Close()
break
} else {
fmt.Println("ok")
}
// sleep 10 seconds and re-send
time.Sleep(10*time.Second)
}
} else {
panic(err)
}
}
Ouput:
> hello
ok
ok
The Client writes to the Server twice. After the first read, the Server closes the connection immediately, but the Client sleeps 10 seconds and then re-writes to the Server with the same already closed connection object(conn).
Why can the second write succeed (returned error is nil)?
Can anyone help?
PS:
In order to check if the buffering feature of the system affects the result of the second write, I edited the Client like this, but it still succeeds:
// Client
if conn, err := net.Dial("tcp", addr); err == nil {
_, err := conn.Write([]byte("hello"))
if err != nil {
fmt.Println(err)
conn.Close()
return
} else {
fmt.Println("ok")
}
// sleep 10 seconds and re-send
time.Sleep(10*time.Second)
b := make([]byte, 400000)
for i := range b {
b[i] = 'x'
}
n, err := conn.Write(b)
if err != nil {
fmt.Println(err)
conn.Close()
return
} else {
fmt.Println("ok", n)
}
// sleep 10 seconds and re-send
time.Sleep(10*time.Second)
} else {
panic(err)
}
And here is the screenshot:
attachment
There are several problems with your approach.
Sort-of a preface
The first one is that you do not wait for the server goroutine
to complete.
In Go, once main() exits for whatever reason,
all the other goroutines still running, if any, are simply
teared down forcibly.
You're trying to "synchronize" things using timers,
but this only works in toy situations, and even then it
does so only from time to time.
Hence let's fix your code first:
package main
import (
"fmt"
"log"
"net"
"time"
)
func main() {
addr := "127.0.0.1:8999"
tcpaddr, err := net.ResolveTCPAddr("tcp4", addr)
if err != nil {
log.Fatal(err)
}
listener, err := net.ListenTCP("tcp", tcpaddr)
if err != nil {
log.Fatal(err)
}
// Server
done := make(chan error)
go func(listener net.Listener, done chan<- error) {
for {
conn, err := listener.Accept()
if err != nil {
done <- err
return
}
go func(conn net.Conn) {
var buffer [1024]byte
n, err := conn.Read(buffer[:])
if err != nil {
log.Println(err)
} else {
log.Println(">", string(buffer[0:n]))
}
if err := conn.Close(); err != nil {
log.Println("error closing server conn:", err)
}
}(conn)
}
}(listener, done)
// Client
conn, err := net.Dial("tcp", addr)
if err != nil {
log.Fatal(err)
}
for i := 0; i < 2; i++ {
_, err := conn.Write([]byte("hello"))
if err != nil {
log.Println(err)
err = conn.Close()
if err != nil {
log.Println("error closing client conn:", err)
}
break
}
fmt.Println("ok")
time.Sleep(2 * time.Second)
}
// Shut the server down and wait for it to report back
err = listener.Close()
if err != nil {
log.Fatal("error closing listener:", err)
}
err = <-done
if err != nil {
log.Println("server returned:", err)
}
}
I've spilled a couple of minor fixes
like using log.Fatal (which is
log.Print + os.Exit(1)) instead of panicking,
removed useless else clauses to adhere to the coding standard of keeping the main
flow where it belongs, and lowered the client's timeout.
I have also added checking for possible errors Close on sockets may return.
The interesting part is that we now properly shut the server down by closing the listener and then waiting for the server goroutine to report back (unfortunately Go does not return an error of a custom type from net.Listener.Accept in this case so we can't really check that Accept exited because we've closed the listener).
Anyway, our goroutines are now properly synchronized, and there is
no undefined behaviour, so we can reason about how the code works.
Remaining problems
Some problems still remain.
The more glaring is you making wrong assumption that TCP preserves
message boundaries—that is, if you write "hello" to the client
end of the socket, the server reads back "hello".
This is not true: TCP considers both ends of the connection
as producing and consuming opaque streams of bytes.
This means, when the client writes "hello", the client's
TCP stack is free to deliver "he" and postpone sending "llo",
and the server's stack is free to yield "hell" to the read
call on the socket and only return "o" (and possibly some other
data) in a later read.
So, to make the code "real" you'd need to somehow introduce these
message boundaries into the protocol above TCP.
In this particular case the simplest approach would be either
using "messages" consisting of a fixed-length and agreed-upon
endianness prefix indicating the length of the following
data and then the string data itself.
The server would then use a sequence like
var msg [4100]byte
_, err := io.ReadFull(sock, msg[:4])
if err != nil { ... }
mlen := int(binary.BigEndian.Uint32(msg[:4]))
if mlen < 0 {
// handle error
}
if mlen == 0 {
// empty message; goto 1
}
_, err = io.ReadFull(sock, msg[5:5+mlen])
if err != nil { ... }
s := string(msg[5:5+mlen])
Another approach is to agree on that the messages do not contain
newlines and terminate each message with a newline
(ASCII LF, \n, 0x0a).
The server side would then use something like
a usual bufio.Scanner loop to get
full lines from the socket.
The remaining problem with your approach is to not dealing with
what Read on a socket returns: note that io.Reader.Read
(that's what sockets implement, among other things) is allowed
to return an error while having had read some data from the
underlying stream. In your toy example this might rightfully
be unimportant, but suppose that you're writing a wget-like
tool which is able to resume downloading of a file: even if
reading from the server returned some data and an error, you
have to deal with that returned chunk first and only then
handle the error.
Back to the problem at hand
The problem presented in the question, I beleive, happens simply because in your setup you hit some TCP buffering problem due to the tiny length of your messages.
On my box which runs Linux 4.9/amd64 two things reliably "fix"
the problem:
Sending messages of 4000 bytes in length: the second call
to Write "sees" the problem immediately.
Doing more Write calls.
For the former, try something like
msg := make([]byte, 4000)
for i := range msg {
msg[i] = 'x'
}
for {
_, err := conn.Write(msg)
...
and for the latter—something like
for {
_, err := conn.Write([]byte("hello"))
...
fmt.Println("ok")
time.Sleep(time.Second / 2)
}
(it's sensible to lower the pause between sending stuff in
both cases).
It's interesting to note that the former example hits the
write: connection reset by peer (ECONNRESET in POSIX)
error while the second one hits write: broken pipe
(EPIPE in POSIX).
This is because when we're sending in chunks worth 4k bytes,
some of the packets generated for the stream manage to become
"in flight" before the server's side of the connection manages
to propagate the information on its closure to the client,
and those packets hit an already closed socket and get rejected
with the RST TCP flag set.
In the second example an attempt to send another chunk of data
sees that the client side already knows that the connection
has been teared down and fails the sending without "touching
the wire".
TL;DR, the bottom line
Welcome to the wonderful world of networking. ;-)
I'd recommend buying a copy of "TCP/IP Illustrated",
read it and experiment.
TCP (and IP and other protocols above IP)
sometimes works not like people expect them to by applying
their "common sense".
I am trying to use https://github.com/astaxie/beego/tree/master/orm to insert a struct into a postgres database. The operation should be simple
import "github.com/astaxie/beego/orm"
type Product struct {
ID string `orm:"pk"`
...
}
product := &Product{ID: productID}
_, err := orm.NewOrm().Insert(product)
if err != nil {
log.Fatal(err)
}
I keep getting this; no LastInsertId available whenever the code runs (the insert is otherwise successful) but I get a crash.
I understand is it due to postgresql limitations because I use https://www.github.com/lib/pq driver.
Is there I way to work around this using beego/orm?
If the crash is being caused by your log.Fatal(err), you can avoid this by checking and avoiding it:
_, err := orm.NewOrm().Insert(product)
if err != nil {
if err.Error() == "no LastInsertId available" {
log.Println(err)
} else {
log.Fatal(err)
}
}
I am trying to connect to two remote mongodb servers using ssh port-forwarding in golang which are used by my graphql server for querying. The intermediary host for the two tunnels is same. So let's say the intermediary host is 123.45.678.678 and the two remote mongodb servers are 1.23.45.67 and 1.23.45.78, I create the tunnels like this,
conn, err := ssh.Dial("tcp", 123.45.678.678, config)
if err != nil {
panic(err)
}
remote1, err := conn.Dial("tcp", "1.23.45.67:27017")
if err != nil {
panic(err)
}
remote2, err := conn.Dial("tcp", "1.23.45.78:27017")
if err != nil {
panic(err)
}
local1, err := net.Listen("tcp", "localhost:27018")
if err != nil {
panic(err)
}
local2, err := net.Listen("tcp", "localhost:27019")
if err != nil {
panic(err)
}
Now i forward traffic from local1 to remote1 and local2 to remote2 like this
go func() {
for {
l, err := local1.Accept()
if err != nil {
panic(err)
}
go func() {
_, err := io.Copy(l, remote1)
if err != nil {
panic(err)
}
}()
go func() {
_, err := io.Copy(remote1, l)
if err != nil {
panic(err)
}
}()
}
}()
go func() {
for {
l, err := local2.Accept()
if err != nil {
panic(err)
}
go func() {
_, err := io.Copy(l, remote2)
if err != nil {
panic(err)
}
}()
go func() {
_, err := io.Copy(remote2, l)
if err != nil {
panic(err)
}
}()
}
}()
And I create two mongo sessions using mgo.Dial and export these two sessions to the graphql whenever this function is called. For some queries ( not all queries, only some complex queries ) which need both the sessions, i see the write: broken pipe error
panic: readfrom tcp 127.0.0.1:27019->127.0.0.1:53128: write tcp 127.0.0.1:27019->127.0.0.1:53128: write: broken pipe
When I debugged this, i figured out that this error occurs when the io.copy happens between l and remote2 in the code snippet above which i guess is due to the disconnection of remote2 tunnel.
The tcpdump showed that the intermediary host is sending the finish flag to the remote2 server after sometime which inturn is leading to the termination of the connection. I am wondering how I can resolve this.