first of all : I'm new to Go, I come from years of java development.
I have developed a little REST API using Gin Gonic.
One of my endpoint occasionally (so I can't reproduce on demand) crashes during an HTTP Get to an external API I don't manage. The error displayed is something like :
stream error: stream ID 4; INTERNAL_ERROR
An extract from the code crashing :
client := &http.Client{}
req, err := http.NewRequest("GET", apiUrl, nil)
if err != nil{
log.Fatal(err)
return result, err
}
resp, err := client.Do(req)
if err != nil {
log.Fatal(err)
return result, err
}
defer resp.Body.Close()
This crashes my server and stops it.
I don't understand what's happening, I'm handling all the errors in the code, so it looks like an uncaught exception comparing to java, but I don't know how to catch that error and keep my server running (I don't care about avoid the error itself, I just want my server to keep going).
In gin-gonic you can use gin.Recover() middleware that helps your application to recover from panic.
You can use instantiation via gin.New() or via gin.Default() (it's already included)
handlers := gin.New()
handlers.Use(gin.Recovery())
log.Fatal makes an exit (dumb me, thanks Volker)
Related
I am trying to achieve functionality similar to Javascript/C#'s async/await. I am trying out the use of semaphores, and found that it works with URLSession in my XCode Playground.
Thus, i am now trying to perform the same thing with Firebase Authentication using the following code:
var response:String? = "test"
let semaphore = DispatchSemaphore(value: 0)
Auth.auth().createUser(withEmail: email, password: password) { (result, err) in
if err != nil {
response = "\(String(describing: err))"
}else{
response = nil
}
semaphore.signal()
}
let _ = semaphore.wait()
if response == nil{
self.transitionToHome()
}
However, the simulator freezes forever, appearing as if the semaphore.signal() never got called. Placing print statements near the semaphore.signal() didn't appear as well. I've also placed the firestore code in a DispatchQueue.global(qos: .background).async and subsequently tried to retrieve the response value in DispatchQueue.main.async but the response did not get updated as well. The code below reflects what i did:
DispatchQueue.global(qos: .background).async {
Auth.auth().createUser(withEmail: email, password: password) { (result, err) in
if err != nil {
response = "\(String(describing: err))"
}else{
response = nil
}
semaphore.signal()
}
let _ = semaphore.wait()
}
if response == nil{
self.transitionToHome()
}
While this did not freeze up the UI, the response value was not picked up after the DispatchQueue was called. I've also called the if-else block within a DispatchQueue.main.async block but that had the same result too.
Further, after waiting for a period of time, i see this error popping up in my xcode terminal:
020-01-02 01:33:25.447842+0800 das-carddeckapp[78136:10508853] Connection 4: received failure notification
2020-01-02 01:33:25.448179+0800 das-carddeckapp[78136:10508853] Connection 4: failed to connect 1:50, reason -1
2020-01-02 01:33:25.448387+0800 das-carddeckapp[78136:10508853] Connection 4: encountered error(1:50)
2020-01-02 01:33:25.457587+0800 das-carddeckapp[78136:10508853] Task <3A3720D6-7549-4C31-96A2-C88B89294821>.<1> HTTP load failed, 0/0 bytes (error code: -1009 [1:50])
I know that using completion handlers will make this work, but I want to try to get this to work before resorting to completion handlers. Any help is greatly appreciated!
The first example (where you’re calling wait on the main thread) is “deadlocking” because it is blocking the main thread with wait, which is to be signaled from the createUser closure, which also wants to run on the main thread (which you’ve blocked).
I know that semaphores is considered a bad pattern but i just wanted to understand why is it not working.
Your second example, where you are dispatching the wait to the global queue resolves the deadlock. But because you’ve dispatched that asynchronously, that means that the main thread will carry on as the global queue waits for the response. As a result, you are almost guaranteed to not have a value ready to return. (And if you attempt to change this to call the global queue synchronously, you’d just re-introduce the deadlock.)
In short, you can’t easily have the main queue wait for an asynchronous method which calls its completion handler on the main queue. That is a programmatic “Catch-22”.
But setting that aside, if this is an iOS app, it just simply a serious problem to ever block the main thread (even if you could solve the deadlock). It’s a horrible UX when an app freezes. This is especially true for unfortunate mobile users who find themselves on poor cellular connection, where it might result in a non-trivial delay. Worse, if you’re unlucky enough to block the main thread at the wrong time, the iOS watchdog process may unceremoniously kill your app.
Hey, I get it. We’ve all been there. When we first encountered asynchronous programming patterns, like network requests, it feels it would be so much more logical and intuitive if we could just wait for the response. But it should be avoided at all costs. You’ll be much happier in the long run if you stick with well-established asynchronous programming patterns, rather than fighting it.
I'm new to Golang, and am stuck on this issue for a day already.
I am trying to write a small code snippet that decrypts (using AWS go sdk) the cipher text which was previously encrypted (using AWS python sdk). I don't see any problem doing this since the encrypted value is just a base64 encoded string, but I keep running into this error, InvalidCiphertextException, when trying to decrypt.
Here's my GO code:
package main
import (
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/kms"
"fmt"
"os"
"github.com/aws/aws-sdk-go/aws"
b64 "encoding/base64"
)
func main() {
// Initialize a session in us-west-2 that the SDK will use to load
// credentials from the shared credentials file ~/.aws/credentials.
sess, err := session.NewSession(&aws.Config{
Region: aws.String("us-east-1")},
)
fmt.Println(err)
// Create KMS service client
svc := kms.New(sess)
ciphertext_regular := "AYADeGDoLfFyXv59h9zNJFR+QcgAXwABABVhd3MtY3J5cHRvLXB1YmxpYy1rZXkAREF4ZFRKNjhiZmM3eTNoUTR3S2JsVjNwMHprNFh6NnVrUVdXRUxvMTFuVEpPRnZLODhTMm5CWUdERm85WU5XeGVaZz09AAEAB2F3cy1rbXMAS2Fybjphd3M6a21zOnVzLWVhc3QtMTo0ODIyMzQ4MTEwMDQ6a2V5L2Y0ZWFhOWZjLThlMTYtNGFkNi1iMjVkLWE2NzM3Y2E2NDg4ZQC4AQIBAHiqDEdiAJHv6/rKQOUfXvXVzNboRyJDQ7oz+NqStcjb1AGvjNtfT8oLIQ9Ju7i5TD0HAAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMhZXqEceZ0lRyt8ehAgEQgDuhJmDyZ9RJS6+lPy7qSarpaXC7v1MVV0EH3r5XVpwbhTMy0/f2Clyu/OMeSfib/n5Kdkm9AjtSDLaiDgIAAAAADAAAEAAAAAAAAAAAAAAAAABOq2YQN4xM3e9IsJVfPIgi/////wAAAAEAAAAAAAAAAAAAAAEAAAAGEqHCS0KXzFEqy+P7DnpM+c82rApjbQBnMGUCMBlbseoO/TU0Ap2UZ8XTO29sErz90jRdyv/k0j/i2NNXKyYrLHSpm5+hUkSokRvgAwIxAMTt3y/x4bzv5GMmNuIoZduElik538c72dEDX43z/AQc/QKjFHm6omhuBVD+sZo9UA=="
data, err1 := b64.StdEncoding.DecodeString(ciphertext_regular)
// Decrypt the data
result1, err1 := svc.Decrypt(&kms.DecryptInput{CiphertextBlob: data})
if err1 != nil {
fmt.Println("Got error decrypting data: ", err1)
os.Exit(1)
}
blob_string := string(result1.Plaintext)
fmt.Println(blob_string)
}
I've tried to look it up online, and it seems like people are doing the same thing as me. (here)
The complete error message:
Got error decrypting data: InvalidCiphertextException:
status code: 400, request id: 35f1ec73-18e3-11e8-9f54-8793fe969339
Anyone has an idea why?
Doug's comment is incorrect.
Anyways, are you setting the encryption context during encryption? If so, this will fail your decryption. Can you post some sample code of your encryption?
The Go dev guide has KMS examples of encrypting/decrypting in Go: https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/kms-example-encrypt-data.html and https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/kms-example-decrypt-blob.html, respectively.
From your posted ciphertext I can see that you encrypted your data with an encryption context (as #Xibz mentions). You need to supply this same encryption context when Decrypt'ing your data. https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/concepts.html#encryption-context
answered
I am having a hard time with Mongodb and Gridfs, using it with Go's http package. I am trying to store a .mp4 file into Gridfs, then pull it out into the browser for playback.
Heres what I am doing now. It successfully pulls the file from
database, I could even write it correctly to a download location.
// Connect to database
// Session to database
func movie(w http.ResponseWriter r *http.Request) {
file, err := db.GridFS("fs").Open("movie.mp4")
if err != nil {
log.Println(err)
}
defer file.Close()
w.Header.Set("Content-type", "video/mp4")
if _, err := io.Copy(w, file); err != nil {
log.Println(err)
}
// I am trying to send it to the browser.
// I want to achieve same thing as, http://localhost/view/movie.mp4,
as if you did that.
}
If the file was on the server, I would just do something like this. But instead I am trying to store it in Mongodb, for all around easier use involving metadata.
func movie(w http.ResponseWriter r *http.Request) {
http.ServeFile("./uploads/movie.mp4") // Easy
}
The browser is receiving something, but its just malformed or corrupted. Just shows the video player with an error message. Any help would be appreciated, I have only been programming a week.
Here is a picture of the error, no console error messages.
Unless someone has an alternative to storing video files for playback
in browser somewhere other than MongoDB or Amazon S3. Please let me know,
thanks.
You may want to check http.ServeContent. It will handle all the mess (content-type, content-length, partial data, cache) automaticlly and save you a lot of time. It takes a ReadSeeker to serve, which GridFile already implented. So your code may simply change to the below.
func movie(w http.ResponseWriter r *http.Request) {
file, err := db.GridFS("fs").Open("movie.mp4")
if err != nil {
log.Println(err)
}
defer file.Close()
http.ServeContent(w,r,"movie.mp4",file.UploadDate(),file)
}
If that doesn't work, please use tools like curl or wget to download the served content and compare it to the orignal one (in db).
I am learning about the best practices involved in REST API design and wrote a function which handles the GET /cities HTTP/1.1 query.
This function contains cities which is a struct array that holds the cityname, citycode of multiple cities.
Below is the code
func FindCitiesHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json;charset=UTF-8")
if len(cities) == 0 {
w.WriteHeader(404)
return
}
if err := json.NewEncoder(w).Encode(cities); err != nil {
/* what to do here? */
}
w.WriteHeader(200)
}
Now when I started thinking about the possible outcomes of this function. I found these situations.
It returns all the cities properly as JSON response successfully. So I return 200 http status.
The list of cities is empty. So there is nothing to return. So I return 404 (resource not found)
It is about to return JSON response of all cities, but something gone wrong during JSON encoding. Now I am confused here, how do I deal with this situation.
I mean how do you convey message properly to user, if
If your business/application logic had some error/exception.
If data access logic found some issues. (say connection to database is not reachable)
Could you guys please help me to suggest best practices you followed in these situations?
200 is correct
404 is probably not correct; A list of cities can be empty, and still exist. (Think: A 0-length array is still an array.) You should probably return 200. You would only return 404 if the list of cities doesn't exist on your server (in other words, that should probably never happen with your API).
If you experience an internal server error, such as with marshaling JSON, you should return an Internal Server Error, status 500.
I'm using huandu/facebook for Golang to access the FB API. https://github.com/huandu/facebook
This works really well locally but when I try to run from the Google App Engine environment, I can't get it to run.
I used this code locally:
res, err := fb.Get("/me", fb.Params{
"fields": "id,first_name,last_name,name",
"access_token": usertoken,
})
In the documentation (link above) they do mention the App Engine environment but I can'f figure out how to ge this to work with the fb.Get convention.
Thanks.
Edit
Almost got it to work!:
// create a global App var to hold app id and secret.
var globalApp = fb.New("<appId>", "<appSecret>")
session := globalApp.Session(usertoken) //User token here
context := appengine.NewContext(r) //Not sure what r should be...
session.HttpClient = urlfetch.Client(context)
res, err := session.Get("/me", nil)
if err := json.NewEncoder(w).Encode(res); err != nil {
panic(err)
}
If I do this I get back the Id and name. Now all I need to do is request the other parameters. Do I do this in the r parameter to the app engine context?
To answer the last question asked, the appengine.NewContext(r) function takes a *http.Request as a parameter, but this refers to the current request, the one your code is executing in. You can use r.URL.Query() if you wanted to get the query parameters that were sent to this request.
If you want to send parameters in another request, like the one to the Facebook API, you can include them directly in the URL you pass to session.Get(). You can use url.Values.Encode if you want to build a query string from a map of values. If you need to make a request using a method other than GET, such as to an API method that expects JSON, you can use http.NewRequest eg.
session.HttpClient = urlfetch.Client(context)
request, err := http.NewRequest("PUT", url, strings.NewReader("{ "someProperty": 1234 }"))
response, err := session.Do(request)