In this post, it is pointed out that response.Body should be closed to avoid resource leak. It is also shown in the overview examples in http package godoc.
In my test code, I send multiple requests to try an API with
resp, err := http.DefaultClient.Do(req)
multiple times in the same function. Is this a bad practice? In this case, do I write defer resp.Body.Close() after each of them, or just once?
url := server.URL + "/ticket/add"
reader = strings.NewReader(`{"id": "test1", "detail": "test1"}`)
req, err := http.NewRequest("POST", url, reader)
assert.Nil(t, err)
resp, err := http.DefaultClient.Do(req)
assert.Nil(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusCreated, resp.StatusCode)
// add a ticket with same id
reader = strings.NewReader(`{"id": "test1"}`)
req, err = http.NewRequest("POST", url, reader)
assert.Nil(t, err)
resp, err = http.DefaultClient.Do(req)
assert.Nil(t, err)
assert.Equal(t, http.StatusInternalServerError, resp.StatusCode)
A related question, on the server side, i.e., inside the func(w http.ResponseWriter, r *http.Request), is it necessary to close the request body as well?
Yes, you need to close both responses. Deferring one call to resp.Body.Close does not somehow effect the other. The *http.Response is different in each case, and they both can be deferred.
On the server side, you do not need to close the Request.Body -- from the http.Request documentation:
// The Server will close the request body. The ServeHTTP
// Handler does not need to.
Bad practice
If you did not reuse resp variable, it would be obvious that you are dealing with different response instances each of which must be closed.
Do sends an HTTP request and returns an HTTP response (instance)...
So, the bad practice is not to do several requests but to reuse the same variable for several responses. It causes code unobviousness. And produces unreachable objects which never be finalized.
Deferred execution
A defer statement pushes a function call onto a list. The list of saved calls is executed after the surrounding function returns.
If you have scheduled single or several deferred executions with reference to the same variable only the last assigned to the variable object's method will be executed (several times for several defers).
Playground with an example
Closing Response.Body
From Response documentation:
It is the caller's responsibility to
close Body.
So, typically you must close each Response.Body.
Garbage collecting and finalization (edit)
Garbage collector invokes bind to collecting objects finalizers to close files, connections and to do other cleanup actions. And there is no finalizers bind to Body object by default.
Your improved snippet:
// ...a lot of code
resp_one, err := http.DefaultClient.Do(req)
// BTW, `assert.Nil` just returns whether the assertion was successful (bool) not terminates a test.
if assert.Nil(t, err) == true {
defer resp_one.Body.Close()
}
// ...a lot of code
resp_two, err = http.DefaultClient.Do(req)
if assert.Nil(t, err) == true {
defer resp_two.Body.Close()
}
Related
The Operator-SDK for Kubernetes Operators has a ctrl.SetControllerReference() function, which claims it enables automatically garbage collecting objects when the managing Custom Resource is deleted.
Sometimes it doesn't seem to delete anything. What did I do wrong?
ctrl.SetControllerReference() only changes metadata.ownerReferences on the local object.
It doesn't automatically call reconciler.Update() for you.
So the changes were never persisted, and the Owner never changed.
The correct way to create an object is to call ctrl.SetControllerReference() before you call reconciler.Create(), e.g.
if err := ctrl.SetControllerReference(mycrd, secret, r.Scheme); err != nil {
log.Error(err, "Failed to set controller reference")
return err
}
log.Info("Creating a new Secret", "Secret.Namespace", secret.Namespace, "Secret.Name", secret.Name)
if err = r.Create(ctx, secret); err != nil {
log.Error(err, "Failed to create new Secret", "Secret.Namespace", secret.Namespace, "Secret.Name", secret.Name)
return err
}
I am playing around with the database/sql package trying to see how it works and understand what would happen if you don't call rows.Close() etc.
So I wrote the following piece of code for inserting a model to database:
func (db Database) Insert(m model.Model) (int32, error) {
var id int32
quotedTableName := m.TableName(true)
// Get insert query
q, values := model.InsertQuery(m)
rows, err := db.Conn.Query(q, values...)
if err != nil {
return id, err
}
for rows.Next() {
err = rows.Scan(&id)
if err != nil {
return id, err
}
}
return id, nil
}
I don't call rows.Close() on purpose to see the consequences. When setting up the database connection I set some properties such as:
conn.SetMaxOpenConns(50)
conn.SetMaxIdleConns(2)
conn.SetConnMaxLifetime(time.Second*60)
Then I attempt to insert 10000 records:
for i := 0; i < 10000; i++ {
lander := models.Lander{
// ...struct fields with random data on each iteration
}
go func() {
Insert(&lander)
}()
}
(It lacks error checking, context timeouts etc. but for the purpose of playing around it gets the job done). When I execute piece of code from above I expect to see at least some errors regarding database connections however the data gets inserted without problems (all 10000 records). When I check the Stats() I see the following:
{MaxOpenConnections:50 OpenConnections:1 InUse:0 Idle:1 WaitCount:9951 WaitDuration:3h9m33.896466243s MaxIdleClosed:48 MaxLifetimeClosed:2}
Since I didn't call rows.Close() I expected to see more OpenConnections or more InUse connections because I am never releasing the connection (maybe I might be wrong, but this is the purpose of Close() to release a Connection and return it to the pool).
So my question is simply what do these Stats() mean and why are there no errors whatsoever when doing the insertion. Also why aren't there more OpenConnections or InUse ones and what are the real consequences of not calling Close()?
According to the docs for Rows:
If Next is called and returns false and there are no further result sets, the Rows are closed automatically and it will suffice to check the result of Err.
Since you iterate all the results, the result set is closed.
First time user of Cadence:
Scenario
I have a cadence server running in my sandbox environment.
Intent is to fetch the workflow status
I am trying to use this cadence client
go.uber.org/cadence/client
on my local host to talk to my sandbox cadence server.
This is my simple code snippet:
var cadClient client.Client
func main() {
wfID := "01ERMTDZHBYCH4GECHB3J692PC" << I got this from cadence-ui
ctx := context.Background()
wf := cadClientlient.GetWorkflow(ctx, wfID,"") <<< Panic hits here
log.Println("Workflow RunID: ",wf.GetID())
}
I am sure getting it wrong because the client does not know how to reach the cadence server.
I referred this https://cadenceworkflow.io/docs/go-client/ to find the correct usage but could not find any reference (possible that I might have missed it).
Any help in how to resolve/implement this, will be of much help
I am not sure what panic you got. Based on the code snippet, it's likely that you haven't initialized the client.
To initialize it, follow the sample code here: https://github.com/uber-common/cadence-samples/blob/master/cmd/samples/common/sample_helper.go#L82
and
https://github.com/uber-common/cadence-samples/blob/aac75c7ca03ec0c184d0f668c8cd0ea13d3a7aa4/cmd/samples/common/factory.go#L113
ch, err := tchannel.NewChannelTransport(
tchannel.ServiceName(_cadenceClientName))
if err != nil {
b.Logger.Fatal("Failed to create transport channel", zap.Error(err))
}
b.Logger.Debug("Creating RPC dispatcher outbound",
zap.String("ServiceName", _cadenceFrontendService),
zap.String("HostPort", b.hostPort))
b.dispatcher = yarpc.NewDispatcher(yarpc.Config{
Name: _cadenceClientName,
Outbounds: yarpc.Outbounds{
_cadenceFrontendService: {Unary: ch.NewSingleOutbound(b.hostPort)},
},
})
if b.dispatcher != nil {
if err := b.dispatcher.Start(); err != nil {
b.Logger.Fatal("Failed to create outbound transport channel: %v", zap.Error(err))
client := workflowserviceclient.New(b.dispatcher.ClientConfig(_cadenceFrontendService))
I am trying to write an HTTP server in Go for school. I am required to AVOID any libraries that make this task easy (the net/http for example).
My problem is that I cannot seem to display the HTTP Response headers correctly. I want to print each header to my terminal, line by line.
I have written this program in Java, and it works well, however, I would like to have this working with a Go program.
I have a function called 'handleClient' that takes an accepted socket.
func handleClient(c net.Conn) {
defer c.Close()
req, _ := bufio.NewReader(c).ReadString('\n')
fmt.Print(string(req))
When using a web browser to connect to 'localhost:8080', my terminal displays "GET / HTTP/1.1", which is correct, however, I need the additional lines to be posted as well. I understand that 'ReadString('\n') is what is stopping this from happening, however, it is the only way I know of to end that line.
How to I start additional lines?
You can call ReadString inside a loop until you hit EOF.
Something like this:
func handleClient(c net.Conn) {
defer c.Close()
r := bufio.NewReader(c)
for {
req, err := r.ReadString('\n')
if err != nil && err != io.EOF {
panic(err)
}
fmt.Println(req)
// break the loop if the err is eof
if err == io.EOF {
break
}
}
mkopriva pointed me in the right direction, however, EOF is not the correct way to break the loop. I need to stop looping when I run into a line that is blank. Technically it is never EOF.
To do this, I have adjusted the if condition to break the loop to the following:
if len(req) <=2 {
After all the headers are read / printed, an HTTP request will end with "\r\n" I believe. I can confirm that the last line is intact of length 2.
Thanks for the help!
1.Can we call send from one thread and recv from another on the same net.UDPConn or net.TCPConn objects?
2.Can we call multiple sends parallely from different threads on the same net.UDPConn or net.TCPConn objects?
I am unable to find a good documentation also for the same.
Is golang socket api thread safe?
I find that it is hard to test if it is thread safe.
Any pointers in the direction will be helpful.
My test code is below:
package main
import (
"fmt"
"net"
"sync"
)
func udp_server() {
// create listen
conn, err := net.ListenUDP("udp", &net.UDPAddr{
IP: net.IPv4(0, 0, 0, 0),
Port: 8080,
})
if err != nil {
fmt.Println("listen fail", err)
return
}
defer conn.Close()
var wg sync.WaitGroup
for i := 0; i < 10; i = i + 1 {
wg.Add(1)
go func(socket *net.UDPConn) {
defer wg.Done()
for {
// read data
data := make([]byte, 4096)
read, remoteAddr, err := socket.ReadFromUDP(data)
if err != nil {
fmt.Println("read data fail!", err)
continue
}
fmt.Println(read, remoteAddr)
fmt.Printf("%s\n\n", data)
// send data
senddata := []byte("hello client!")
_, err = socket.WriteToUDP(senddata, remoteAddr)
if err != nil {
return
fmt.Println("send data fail!", err)
}
}
}(conn)
}
wg.Wait()
}
func main() {
udp_server()
}
Is it OK for this test code?
The documentation for net.Conn says:
Multiple goroutines may invoke methods on a Conn simultaneously.
Multiple goroutines may invoke methods on a Conn simultaneously.
My interpretation of the doc above, is that nothing catastrophic will happen if you invoke Read and Write on a net.Conn from multiple go routines, and that calls to Write on a net.Conn from multiple go routines will be serialised so that the bytes from 2 separate calls to Write will not be interleaved as they are written to the network.
The problem with the code you have presented is that there is no guarantee that Write will write the whole byte slice provided to it in one go. You are ignoring the indication of how many bytes have been written.
_, err = socket.WriteToUDP(senddata, remoteAddr)
So to make sure you write everything you would need to loop and call Write till all the senddata is sent. But net.Conn only ensures that data from a single call to Write is not interleaved. Given that you could be sending a single block of data with multiple calls to write there is no guarantee that the single block of data would reach its destination intact.
So for example 3 "hello client!" messages could arrive in the following form.
"hellohellohello client! client! client!"
So if you want reliable message writing on a net.Conn from multiple go routines you will need to synchronise those routines to ensure that single messages are written intact.
If I wanted to do this, as a first attempt I would have a single go routine reading from one or many message channels and writing to a net.Conn and then multiple go routines can write to those message channels.