Go HTTP Request with Basic Auth returning a 401 instead of a 301 redirect - redirect

Question

Using Go 1.5.1.
When I try to make a request to a site that automatically redirects to HTTPS using Basic Auth I would expect to get a 301 Redirect response, instead I get a 401.
package main
import "net/http"
import "log"
func main() {
url := "http://aerolith.org/files"
username := "cesar"
password := "password"
req, err := http.NewRequest("GET", url, nil)
if err != nil {
log.Println("error", err)
}
if username != "" || password != "" {
req.SetBasicAuth(username, password)
log.Println("[DEBUG] Set basic auth to", username, password)
}
cli := &http.Client{
}
resp, err := cli.Do(req)
if err != nil {
log.Println("Do error", err)
}
log.Println("[DEBUG] resp.Header", resp.Header)
log.Println("[DEBUG] req.Header", req.Header)
log.Println("[DEBUG] code", resp.StatusCode)
}
Note that curl returns a 301:
curl -vvv http://aerolith.org/files --user cesar:password
Any idea what could be going wrong?

Answer 1

A request to http://aerolith.org/files redirects to https://aerolith.org/files (note change from http to https). A request to https://aerolith.org/files redirects to https://aerolith.org/files/ (note addition of trailing /).
Curl does not follow redirects. Curl prints the 301 status for the redirect from http://aerolith.org/files to https://aerolith.org/files/.
The Go client follows the two redirects to https://aerolith.org/files/. The request to https://aerolith.org/files/ returns with status 401 because the Go client does not propagate the authorization header through the redirects.
Requests to https://aerolith.org/files/ from the Go client and Curl return status 200.
If you want to follow the redirects and auth successfully, set auth header in a CheckRedirect function:
cli := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
if len(via) >= 10 {
return errors.New("stopped after 10 redirects")
}
req.SetBasicAuth(username, password)
return nil
}}
resp, err := cli.Do(req)
If you want to match what Curl does, use a transport directly. The transport does not follow redirects.
resp, err := http.DefaultTransport.RoundTrip(req)
The application can also use the client CheckRedirect function and a distinguished error to prevent redirects as shown in an answer to How Can I Make the Go HTTP Client NOT Follow Redirects Automatically?. This technique seems to be somewhat popular, but is more complicated than using the transport directly.
redirectAttemptedError := errors.New("redirect")
cli := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return redirectAttemptedError
}}
resp, err := cli.Do(req)
if urlError, ok := err.(*url.Error); ok && urlError.Err == redirectAttemptedError {
// ignore error from check redirect
err = nil
}
if err != nil {
log.Println("Do error", err)
}