POST data faild using http.NewRequest - forms

I am trying to pass data from one golang service to another using http.NewRequest(). To do it I used following code:
httpClient := http.Client{}
userserviceUrl := "http://user:7071/checkemail"
form := url.Values{}
form.Set("uuid", uuid)
form.Set("email", email)
b := bytes.NewBufferString(form.Encode())
req, err := http.NewRequest("POST", userserviceUrl, b)
if err != nil {
log.Println(err)
}
opentracing.GlobalTracer().Inject(
validateEmailSpan.Context(),
opentracing.HTTPHeaders,
opentracing.HTTPHeadersCarrier(req.Header))
resp, err := httpClient.Do(req)
//_, err = http.PostForm("http://user:7071/checkemail", url.Values{"uuid": {uuid}, "email": {email}})
if err != nil {
log.Println("Couldnt verify email address user service sends an error : ", err)
}
defer resp.Body.Close()
I got this from Golang: http.NewRequest POST
When I try to dump received data from user service:
req.ParseForm()
log.Println("Form values : ", req.Form)
I get an empty map[]
Here I just try to inject tracing span to my request, previously I have used http.PostForm() to pass data, it worked perfectly. But I have no idea to pass tracing to it.

From the docs for ParseForm:
[...] when the Content-Type is not application/x-www-form-urlencoded, the request Body is not read, and r.PostForm is initialized to a non-nil, empty value.
PostForm sets the Content-Type automatically, but now you have to do it yourself:
req, err := http.NewRequest("POST", userserviceUrl, strings.NewReader(form.Encode()))
// TODO: handle error
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")

Related

Go Echo: POST Method gives Error "Method not allowed"

Building an app with echo and basically created some routes.
The GET ones are working fine, but the post one is give me the error:
Do not really understand where the error lies here.
{...."method":"GET","uri":"/addPerson", message=Method Not Allowed","...."bytes_in":0,"bytes_out":33}
main.go snippet
func initEchoServer() {
e := echo.New()
e.Use(middleware.Logger())
e.Use(middleware.Recover())
// get all persons
e.GET("/persons", Info)
// get specific id
e.GET("/persons/:id", getPerson)
e.POST("/addPerson", addPerson)
e.Logger.Fatal(e.Start(viper.GetString("port")))
}
func addPerson(c echo.Context) error {
ctx := context.Background()
db, err := sql.Open("postgres", "host=postgres port=5432 user=postgres dbname=postgres password=postgres sslmode=disable")
if err != nil {
log.Fatal(err)
}
queries := postgres.New(db)
insertedPerson, err := queries.CreatePersons(ctx, postgres.CreatePersonsParams{
Firstname: "Mike",
Lastname: "Jordan",
})
if err != nil {
log.Errorf("Failed to insert a person %v", err)
return err
}
fmt.Println(insertedPerson)
return c.JSONPretty(http.StatusOK, insertedPerson, " ")
}
queries.sql.go snippet
type CreatePersonsParams struct {
Firstname string
Lastname string
}
func (q *Queries) CreatePersons(ctx context.Context, arg CreatePersonsParams) (Person, error) {
row := q.db.QueryRowContext(ctx, createPersons, arg.Firstname, arg.Lastname)
var i Person
err := row.Scan(&i.ID, &i.Firstname, &i.Lastname)
return i, err
}
you're use post method in routers
e.POST("/addPerson", addPerson)
You can use postman to hit API using POST method, don't use browser
If you register routes with POST in echo, it will only register POST method on that path. But it seems that you GET that path.
You can use e.GET().

ObjectIdFromHex invalid byte error on identical strings

I'm trying to implement a FindOne method in my Golang REST API. The trouble comes where i have to search by ID. I have to convert the ID into something readable by the database, so i use primitive.ObjectIDFromHex(id)
The problem is that this method throws an error :
2021/06/19 06:56:15 encoding/hex: invalid byte: U+000A
ONLY when i call it with the id that comes from my URL GET params.
I did two versions : one with hard-coded ID, and one with GET ID. See code below.
func Admin(id string) (bson.M, error) {
coll, err := db.ConnectToCollection("admin")
if err != nil {
log.Fatal(err)
}
var admin bson.M
HardCoded := "60cb275c074ab46a1aeda45e"
fmt.Println(HardCoded) // Just to be sure : the two strings seem identical
fmt.Println(id)
objetId, err := primitive.ObjectIDFromHex(id) // throws encoding error
// objetId, err := primitive.ObjectIDFromHex(HardCoded) // Doesnt throw encoding err
if err != nil {
log.Fatal(err)
}
var ctx = context.TODO()
if err := coll.FindOne(ctx, bson.M{"_id": objetId}).Decode(&admin); err != nil {
log.Fatal(err)
}
return admin, nil
}
Of course, you'll want to know where the param id comes from.
Here you go :
func GetAdmin(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
params := mux.Vars(r)
admin, err := Admin(params["id"]) // Calling the Admin function above
if err != nil {
fmt.Println(err)
http.Error(w, err.Error(), http.StatusUnauthorized)
} else {
JSON, err := json.Marshal(admin)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
w.Write(JSON)
}
}
Trim the line feed from the end of id:
id = strings.TrimSpace(id)
Use the %q format verb when debugging issues like this. The line feed is clearly visible in this output:
fmt.Printf("%q\n", HardCoded) // prints "60cb275c074ab46a1aeda45e"
fmt.Printf("%q\n", id) // prints "60cb275c074ab46a1aeda45e\n"

Create a broken form with http.Client

How to create a 'broken' form for testing purposes with http.Client that will trigger a response error on ParseForm() in a handler function?
I have the following code:
func (app *App) signupUser(w http.ResponseWriter, r *http.Request) {
err := r.ParseForm()
if err != nil { // how to trigger this if statement?
sendStatus(w, http.StatusBadRequest, "error parsing form")
return
}
...
Then during testing I'm using http.Client to test this controller:
resp, err = client.PostForm(ts.URL+"/user/signup", url.Values{
"email": {testEmail},
"password": {goodPassword},
)
if err != nil {
t.Error(err) // I never get here
}
First of all client.PostForm will do a proper job of populating the post form, so you have to populate it manually.
Then if you dig into the call stack of r.ParseForm(), you'll eventually arrive to the error producers (barring reading issues, etc.):
"net/url".QueryUnescape which "returns an error if any % is not followed by two hexadecimal digits."
So, for example:
r := &http.Request{
// ensures it reads the body
Method: "POST",
// string with bad url encoding '%3z'
Body: io.NopCloser(strings.NewReader("foo%3z1%26bar%3D2")),
// ensures it parses the body as post form
Header: map[string][]string{
"Content-Type": []string{
"application/x-www-form-urlencoded",
},
},
}
err := r.ParseForm()
if err == nil {
panic("no err")
}
// err -> invalid URL escape "%3z"

Go web service - POST tar.gz file as request body

I need to implement web service in go that processes tar.gz files and I wonder what is the correct way, what content type I need to define, etc.
plus, I found that a lot of things are handled automatically - on the client side I just post a gzip reader as request body and Accept-Encoding: gzip header is added automatically, and on the server side - I do not need to gunzip the request body, it is already extracted to tar. does that make sense?
Can I rely that it would be like this with any client?
Server:
func main() {
router := mux.NewRouter().StrictSlash(true)
router.Handle("/results", dataupload.NewUploadHandler()).Methods("POST")
log.Fatal(http.ListenAndServe(*address, router))
}
Uploader:
package dataupload
import (
"errors"
log "github.com/Sirupsen/logrus"
"io"
"net/http"
)
// UploadHandler responds to /results http request, which is the result-service rest API for uploading results
type UploadHandler struct {
uploader Uploader
}
// NewUploadHandler creates UploadHandler instance
func NewUploadHandler() *UploadHandler {
return &UploadHandler{
uploader: TarUploader{},
}
}
func (uh UploadHandler) ServeHTTP(writer http.ResponseWriter, request *http.Request) {
retStatus := http.StatusOK
body, err := getBody(request)
if err != nil {
retStatus = http.StatusBadRequest
log.Error("Error fetching request body. ", err)
} else {
_, err := uh.uploader.Upload(body)
}
writer.WriteHeader(retStatus)
}
func getBody(request *http.Request) (io.ReadCloser, error) {
requestBody := request.Body
if requestBody == nil {
return nil, errors.New("Empty request body")
}
var err error
// this part is commented out since somehow the body is already gunzipped - no need to extract it.
/*if strings.Contains(request.Header.Get("Accept-Encoding"), "gzip") {
requestBody, err = gzip.NewReader(requestBody)
}*/
return requestBody, err
}
Client
func main() {
f, err := os.Open("test.tar.gz")
if err != nil {
log.Fatalf("error openning file %s", err)
}
defer f.Close()
client := new(http.Client)
reader, err := gzip.NewReader(f)
if err != nil {
log.Fatalf("error gzip file %s", err)
}
request, err := http.NewRequest("POST", "http://localhost:8080/results", reader)
_, err = client.Do(request)
if err != nil {
log.Fatalf("error uploading file %s", err)
}
}
The code you've written for the client is just sending the tarfile directly because of this code:
reader, err := gzip.NewReader(f)
...
request, err := http.NewRequest("POST", "http://localhost:8080/results", reader)
If you sent the .tar.gz file content directly, then you would need to gunzip it on the server. E.g.:
request, err := http.NewRequest(..., f)
I think that's closer to the behavior you should expect third-party clients to exhibit.
Claerly not, but maybe...
Golang provides a very good support for the http client (and server). This is one of the first language to support http2 and the design of the API clearly shows their concern on having a fast http.
This is why they add Accept-Econding: gzip automatically. That will dramatically reduce the size of the server response and then optimize the transfer.
But the gzip remains an option in http 1 and not all of the client will push this header to your server.
Note that the Content-Type describes the type of data you are sending (here a tar.gz but could be application/json, test/javascript, ...), when the Accept-Encoding describes the way the data has been encoded for the transport
Go will take care of transparently handling the Accept-Encoding for you because it is responsible of the transport of the data. Then it will be up to you to handle the Content-Type because only you know how to give a sense to the content you received

How to read from request then use that result to do POST request then process its results

I'm trying to read from request then use that result to do POST request to another endpoint then process its results then return its results in JSON.
I have below code so far:
// POST
func (u *UserResource) authenticate(request *restful.Request, response *restful.Response) {
Api := Api{url: "http://api.com/api"}
usr := new(User)
err := request.ReadEntity(&usr)
if err != nil {
response.WriteErrorString(http.StatusInternalServerError, err.Error())
return
}
api_resp, err := http.Post(Api.url, "text/plain", bytes.NewBuffer(usr))
if err != nil {
response.WriteErrorString(http.StatusInternalServerError, err.Error())
return
}
defer api_resp.Body.Close()
body, err := ioutil.ReadAll(api_resp.Body)
response.WriteHeader(http.StatusCreated)
err = xml.Unmarshal(body, usr)
if err != nil {
fmt.Printf("error: %v", err)
return
}
// result, err := json.Marshal(usr)
// response.Write(result)
response.WriteEntity(&usr)
fmt.Printf("Name: %q\n", usr.UserName)
}
I'm using Go Restful package for Writes and Reads.
I'm getting this error when I compile the file:
src\login.go:59: cannot use usr (type *User) as type []byte in argument to bytes.NewBuffer
What would be the best way to solve this issue so I can do a POST with payload correctly?
You need to marshal your data structure to slice of bytes. Something like this:
usrXmlBytes, err := xml.Marshal(usr)
if err != nil {
response.WriteErrorString(http.StatusInternalServerError, err.Error())
return
}
api_resp, err := http.Post(Api.url, "text/plain", bytes.NewReader(usrXmlBytes))
http.Post takes an io.Reader as the third argument. You could implement io.Reader on your User type or more simply serialize your data and use the bytes pkg to to implement io.Reader
b, err := json.Marshal(usr)
if err != nil {
response.WriteErrorString(http.StatusInternalServerError, err.Error())
return
}
api_resp, err := http.Post(Api.url, "text/plain", bytes.NewReader(b))