Problem: Although I can easily issue GET and POST commands from curl on my local docker socket, when I try to do the POST equivalent in Golang for a docker pull action, using net.Dial, I see no action taken on Docker's behalf.
Note that, meanwhile, the GET action works just fine using docker sockets via the golang client.
For example, when running the code at the bottom of this post, I see:
2018/01/05 14:16:33 Pulling http://localhost/v1.24/images/create?fromImage=fedora&tag=latest ......
2018/01/05 14:16:34 Succeeded pull for http://localhost/v1.24/images/create?fromImage=fedora&tag=latest &{200 OK 200 HTTP/1.1 1 1 map[Docker-Experimental:[true] Ostype:[linux] Server:[Docker/17.09.0-ce (linux)] Api-Version:[1.32] Content-Type:[application/json] Date:[Fri, 05 Jan 2018 19:16:34 GMT]] 0xc42010a100 -1 [chunked] false false map[] 0xc420102000 <nil>}
The code for attempting to do a pull via a POST action:
// pullImage is the equivalent of curl --unix-socket /var/run/docker.sock -X POST http://localhost/images/create?fromImage=alpine
func pullImage(img string, tag string) (err error) {
fd := func (proto, addr string) (conn net.Conn, err error) {
return net.Dial("unix", "/var/run/docker.sock")
}
tr := &http.Transport{
Dial: fd,
}
client := &http.Client{Transport: tr}
imageUrl := fmt.Sprintf("http://localhost/v1.24/images/create?fromImage=%s&tag=%s", img, tag)
log.Printf("Pulling %s ...... ", imageUrl)
resp, err := client.Post(imageUrl, "application/json", nil)
if resp.StatusCode == 200 && err == nil {
log.Printf("Succeeded pull for %s %v",imageUrl, resp)
} else {
log.Printf("FAILED pull for %s , ERROR = (( %s )) ",imageUrl , err)
}
return err
Question:
What does curl --unix-socket /var/run/docker.sock -X POST http://localhost/v1.24/images/create?fromImage=fedora do differently then the Golang code in the above snippet?
I figured this out. Actually, my prolem was that I wasn't completing the response body.
Adding
defer resp.Body.Close()
after the rest, err := client... clause was enough to finish triggering an entire pull. Not quite sure why closing the response body effects the ability of docker to start pulling the images down, but in any case, that was the case.
Related
sql.Open() wouldn't error:
if db, err = sql.Open("postgres", url); err != nil {
return nil, fmt.Errorf("Postgres connect error : (%v)", err)
}
but db.Ping() would error:
if err = db.Ping(); err != nil {
return nil, fmt.Errorf("Postgres ping error : (%v)", err)
}
and it was simply because the lib/pq connection string wouldn't connect from within a docker container with the seperated connection parameters.
For example:
url := fmt.Sprintf("user=%v password=%v host=%v port=%v dbname=%v",
rs.conf.Redshift.User,
rs.conf.Redshift.Password,
rs.conf.Redshift.Host,
rs.conf.Redshift.Port,
rs.conf.Redshift.DB)
Using the connection string as a URL worked:
url := fmt.Sprintf("postgres://%v:%v#%v:%v/%v?sslmode=disable",
pql.conf.Postgres.User,
pql.conf.Postgres.Password,
pql.conf.Postgres.Host,
pql.conf.Postgres.Port,
pql.conf.Postgres.DB)
See lib/pq docs here:
https://godoc.org/github.com/lib/pq
I was stuck on this for more than a day and I owe the fix to Nikolay Sandalov's comment here in GitHub:
https://github.com/coreos/clair/issues/134#issuecomment-491300639
Thank you, Nikolay 🙇🏻♂️
I've written a simple go code that, sends a GET request to an API and in response I receive a 401 error. However when I use cURL, I receive the desired response. I also get expected response using API Tester. So, I believe, there has to be something wrong with my code and that, I'm unable to find out.
Below is my Go code, that, responds with 401 Error
func main() {
clusterId := os.Getenv("CLUSTER_ID")
apiUrl := "https://api.qubole.com/api/v1.3/clusters/"+clusterId+"/state"
auth_token := os.Getenv("X_AUTH_TOKEN")
fmt.Println("URL - ",apiUrl)
req, err := http.NewRequest("GET", apiUrl, nil)
if(err != nil){
fmt.Println("ERROR in getting rest response",err)
}
req.Header.Set("X-Auth-Token", auth_token)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
res, err := http.DefaultClient.Do(req)
if(err != nil){
fmt.Println("Error: No response received", err)
}
defer res.Body.Close()
//print raw response body for debugging purposes
body, _ := ioutil.ReadAll(res.Body)
fmt.Println(string(body))
}
Extract of Response/Error I get, is as follows:
URL - https://api.qubole.com/api/v1.3/clusters/presto-bi/state
{"error":{"error_code":401,"error_message":"Invalid Token"}}
Now, Following is the cURL command that, returns me the desired response
curl -X GET -H "X-AUTH-TOKEN:$X_AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json" "https://us.qubole.com/api/v1.3/clusters/presto-bi/state"
below is the stdout I receive, which is as expected: {"state":"DOWN"}%
Need check api hostname at golang and curl again. Thanks!
The error is because, the documentation of API provider states the host WRONG (API Documentation) . But, since the portal login is us.qubole.com (PORTAL Login URL), cURL command was written considering that in mind.
I'm just guessing here without enough information. I'm assuming clusterId := os.Getenv("CLUSTER_ID") is presto-bi. If that is the case, then you are just missing "clusters" in your path.
apiUrl := "https://api.qubole.com/api/v1.3/clusters/"+clusterId+"/state"
Also, shouldn't you use us.qubole.com/api instead of api.qubole.com?
There are two go apps, one is stapi listening on port 8050 and providing RESTful APIs, another is client to consume those APIs.
Both are running on different servers, client is often getting error when calling APIs with HTTP POST method. Below are few lines from client log (real IP replaced with imaginary one)
2018/02/17 11:42:58 ERROR: [DoLogin] API Error: [Post https://123.123.123.123:8050/v1/st/verifyuser: dial tcp 123.123.123.123:8050: getsockopt: connection refused]
2018/02/17 11:47:14 ERROR: [CreateAttempt] Error: [Post https://123.123.123.123:8050/v1/userattempts/createattempt: dial tcp 123.123.123.123:8050: getsockopt: connection refused]
It is intermittent and making the app unreliable, out of approx 1k request i got such error for approx 50+ request.
Initially stapi was listening on all IPs
httpSrv := http.Server{
Addr: ":8050",
Handler: router, // < gin router
...
}
But after reading the workaroung in Golang HTTP Post error: connection refused i modified the stapi app and make it listening on different IPs, as shown below
$ sudo lsof -i -P -n | grep LISTEN
stapi 4775 samtech 10u IPv4 2388179 0t0 TCP 123.123.123.123:8050 (LISTEN)
stapi 4775 samtech 11u IPv6 2388181 0t0 TCP [::1]:8050 (LISTEN)
stapi 4775 samtech 12u IPv4 2388183 0t0 TCP 127.0.0.1:8050 (LISTEN)
But still the issue is same, what else i should check and fix ? Please suggest.
API is protected with JWT, here is how client is making POST requests
func (w *OST) DoLogin(c *gin.Context) {
...
ud := stapimodels.UserLogin{}
err := c.BindJSON(&ud)
...
//call api to save user response
url := config.AppConfig.APIBaseURL + "st/verifyuser"
res, err := api.JwtApi.APIPost(url, &ud)
if err != nil {
g.Logger.Errorm("DoLogin", "Error: %v", err)
t.Error("Error", err.Error())
return
}
...
}
//APIPost - call given apiurl with POST method and pass data
func (j *JwtAPI) APIPost(apiurl string, postdata interface{}) (*APIResult, error) {
if postdata == nil {
return nil, fmt.Errorf("postdata is nil")
}
jsondata, err := toJSON(postdata)
if err != nil {
return nil, err
}
resp, err := j.makeRequest(http.MethodPost, apiurl, jsondata)
if err != nil {
return nil, err
}
defer resp.Body.Close()
res := APIResult{}
json.NewDecoder(resp.Body).Decode(&res)
return &res, nil
}
//makeRequest makes http request for given url with given method
// also inject Authorization Header
func (j *JwtAPI) makeRequest(method, apiurl string, body io.Reader) (*http.Response, error) {
retry := 0
//Create []byte buffer from body - so it can be passed in further retries
var buf []byte
if body != nil {
buf, _ = ioutil.ReadAll(body)
}
r, err := http.NewRequest(method, apiurl, bytes.NewReader(buf))
if err != nil {
return nil, err
}
r.Header.Set("Authorization", "bearer "+j.token.AccessToken)
r.Header.Set("Content-Type", "application/json")
client := j.getClient()
resp, err := client.Do(r)
if err != nil {
return nil, err
}
return resp, nil
}
func (j *JwtAPI) getClient() *http.Client {
// default timeout (if not set by client)
timeoutInSec := 10
if j.Timeout.Seconds() > 0 {
// client sets timeout, so use it
timeoutInSec = int(j.Timeout.Seconds())
}
client := &http.Client{
Timeout: time.Second * time.Duration(timeoutInSec),
}
return client
}
To make your code more resilient you should add some retries with back-offs, so even when the connection was refused it is still working.
Connection refused means that the port is not opened. Is there any firewall or proxies in between? The authentication part shouldn't matter here because it doesn't even get to this point.
Some things that you can check:
Make sure the service is running
Check for firewall configuration
Implement retries for resilience
Is the IP-Address fixed? Is Dynamic DNS used and maybe not updated?
Package for back-off retrying
As for implementing the back-off you might try this package:
https://github.com/cenkalti/backoff
It is listing examples on how to use it and it's pretty much exactly what you need:
// An operation that may fail.
operation := func() error {
// do the request here and check the response code (or check the response body depending on your need) . e.g. above 500 should retry, above 400 and below 500, it should be a client side error and retrying might not help much
return nil // or an error
}
err := Retry(operation, NewExponentialBackOff())
if err != nil {
// Handle error.
return
}
// Operation is successful.
in GO net/http Response Body annotation says:
It is the caller's responsibility to close Body. The default HTTP client's Transport does not attempt to reuse HTTP/1.0 or HTTP/1.1 TCP connections ("keep-alive") unless the Body is read to completion and is
closed.
It's mean: if I use http.Get and don't call resp.Body.Close() then it will not resue HTTP/1.0 or HTTP/1.1 TCP connections ("keep-alive") yeah?
so I write some code:
package main
import (
"time"
"fmt"
"io/ioutil"
"net/http"
)
func main() {
resp, err := http.Get("http://127.0.0.1:8588")
if err != nil {
panic(err)
}
_, err = ioutil.ReadAll(resp.Body)
if err != nil {
panic(err)
}
resp2, err := http.Get("http://127.0.0.1:8588")
if err != nil {
panic(err)
}
_, err = ioutil.ReadAll(resp2.Body)
if err != nil {
panic(err)
}
fmt.Println("before time sleep")
time.Sleep(time.Second * 35)
}
and I only see ONE tcp connection build in wireshark, why?
I don't close res.Body so the http client should't be reuse the tcp connection.
this problem has been solved in
https://github.com/golang/go/issues/22954.
You have read it till the end in first occurence of line:
_, err = ioutil.ReadAll(resp.Body)
So the connection is ready to be resused. Try not to read and run again.
Using Go 1.5.1.
When I try to make a request to a site that automatically redirects to HTTPS using Basic Auth I would expect to get a 301 Redirect response, instead I get a 401.
package main
import "net/http"
import "log"
func main() {
url := "http://aerolith.org/files"
username := "cesar"
password := "password"
req, err := http.NewRequest("GET", url, nil)
if err != nil {
log.Println("error", err)
}
if username != "" || password != "" {
req.SetBasicAuth(username, password)
log.Println("[DEBUG] Set basic auth to", username, password)
}
cli := &http.Client{
}
resp, err := cli.Do(req)
if err != nil {
log.Println("Do error", err)
}
log.Println("[DEBUG] resp.Header", resp.Header)
log.Println("[DEBUG] req.Header", req.Header)
log.Println("[DEBUG] code", resp.StatusCode)
}
Note that curl returns a 301:
curl -vvv http://aerolith.org/files --user cesar:password
Any idea what could be going wrong?
A request to http://aerolith.org/files redirects to https://aerolith.org/files (note change from http to https). A request to https://aerolith.org/files redirects to https://aerolith.org/files/ (note addition of trailing /).
Curl does not follow redirects. Curl prints the 301 status for the redirect from http://aerolith.org/files to https://aerolith.org/files/.
The Go client follows the two redirects to https://aerolith.org/files/. The request to https://aerolith.org/files/ returns with status 401 because the Go client does not propagate the authorization header through the redirects.
Requests to https://aerolith.org/files/ from the Go client and Curl return status 200.
If you want to follow the redirects and auth successfully, set auth header in a CheckRedirect function:
cli := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
if len(via) >= 10 {
return errors.New("stopped after 10 redirects")
}
req.SetBasicAuth(username, password)
return nil
}}
resp, err := cli.Do(req)
If you want to match what Curl does, use a transport directly. The transport does not follow redirects.
resp, err := http.DefaultTransport.RoundTrip(req)
The application can also use the client CheckRedirect function and a distinguished error to prevent redirects as shown in an answer to How Can I Make the Go HTTP Client NOT Follow Redirects Automatically?. This technique seems to be somewhat popular, but is more complicated than using the transport directly.
redirectAttemptedError := errors.New("redirect")
cli := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return redirectAttemptedError
}}
resp, err := cli.Do(req)
if urlError, ok := err.(*url.Error); ok && urlError.Err == redirectAttemptedError {
// ignore error from check redirect
err = nil
}
if err != nil {
log.Println("Do error", err)
}