answered
I am having a hard time with Mongodb and Gridfs, using it with Go's http package. I am trying to store a .mp4 file into Gridfs, then pull it out into the browser for playback.
Heres what I am doing now. It successfully pulls the file from
database, I could even write it correctly to a download location.
// Connect to database
// Session to database
func movie(w http.ResponseWriter r *http.Request) {
file, err := db.GridFS("fs").Open("movie.mp4")
if err != nil {
log.Println(err)
}
defer file.Close()
w.Header.Set("Content-type", "video/mp4")
if _, err := io.Copy(w, file); err != nil {
log.Println(err)
}
// I am trying to send it to the browser.
// I want to achieve same thing as, http://localhost/view/movie.mp4,
as if you did that.
}
If the file was on the server, I would just do something like this. But instead I am trying to store it in Mongodb, for all around easier use involving metadata.
func movie(w http.ResponseWriter r *http.Request) {
http.ServeFile("./uploads/movie.mp4") // Easy
}
The browser is receiving something, but its just malformed or corrupted. Just shows the video player with an error message. Any help would be appreciated, I have only been programming a week.
Here is a picture of the error, no console error messages.
Unless someone has an alternative to storing video files for playback
in browser somewhere other than MongoDB or Amazon S3. Please let me know,
thanks.
You may want to check http.ServeContent. It will handle all the mess (content-type, content-length, partial data, cache) automaticlly and save you a lot of time. It takes a ReadSeeker to serve, which GridFile already implented. So your code may simply change to the below.
func movie(w http.ResponseWriter r *http.Request) {
file, err := db.GridFS("fs").Open("movie.mp4")
if err != nil {
log.Println(err)
}
defer file.Close()
http.ServeContent(w,r,"movie.mp4",file.UploadDate(),file)
}
If that doesn't work, please use tools like curl or wget to download the served content and compare it to the orignal one (in db).
Related
first of all : I'm new to Go, I come from years of java development.
I have developed a little REST API using Gin Gonic.
One of my endpoint occasionally (so I can't reproduce on demand) crashes during an HTTP Get to an external API I don't manage. The error displayed is something like :
stream error: stream ID 4; INTERNAL_ERROR
An extract from the code crashing :
client := &http.Client{}
req, err := http.NewRequest("GET", apiUrl, nil)
if err != nil{
log.Fatal(err)
return result, err
}
resp, err := client.Do(req)
if err != nil {
log.Fatal(err)
return result, err
}
defer resp.Body.Close()
This crashes my server and stops it.
I don't understand what's happening, I'm handling all the errors in the code, so it looks like an uncaught exception comparing to java, but I don't know how to catch that error and keep my server running (I don't care about avoid the error itself, I just want my server to keep going).
In gin-gonic you can use gin.Recover() middleware that helps your application to recover from panic.
You can use instantiation via gin.New() or via gin.Default() (it's already included)
handlers := gin.New()
handlers.Use(gin.Recovery())
log.Fatal makes an exit (dumb me, thanks Volker)
I'm new to Golang, and am stuck on this issue for a day already.
I am trying to write a small code snippet that decrypts (using AWS go sdk) the cipher text which was previously encrypted (using AWS python sdk). I don't see any problem doing this since the encrypted value is just a base64 encoded string, but I keep running into this error, InvalidCiphertextException, when trying to decrypt.
Here's my GO code:
package main
import (
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/kms"
"fmt"
"os"
"github.com/aws/aws-sdk-go/aws"
b64 "encoding/base64"
)
func main() {
// Initialize a session in us-west-2 that the SDK will use to load
// credentials from the shared credentials file ~/.aws/credentials.
sess, err := session.NewSession(&aws.Config{
Region: aws.String("us-east-1")},
)
fmt.Println(err)
// Create KMS service client
svc := kms.New(sess)
ciphertext_regular := "AYADeGDoLfFyXv59h9zNJFR+QcgAXwABABVhd3MtY3J5cHRvLXB1YmxpYy1rZXkAREF4ZFRKNjhiZmM3eTNoUTR3S2JsVjNwMHprNFh6NnVrUVdXRUxvMTFuVEpPRnZLODhTMm5CWUdERm85WU5XeGVaZz09AAEAB2F3cy1rbXMAS2Fybjphd3M6a21zOnVzLWVhc3QtMTo0ODIyMzQ4MTEwMDQ6a2V5L2Y0ZWFhOWZjLThlMTYtNGFkNi1iMjVkLWE2NzM3Y2E2NDg4ZQC4AQIBAHiqDEdiAJHv6/rKQOUfXvXVzNboRyJDQ7oz+NqStcjb1AGvjNtfT8oLIQ9Ju7i5TD0HAAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMhZXqEceZ0lRyt8ehAgEQgDuhJmDyZ9RJS6+lPy7qSarpaXC7v1MVV0EH3r5XVpwbhTMy0/f2Clyu/OMeSfib/n5Kdkm9AjtSDLaiDgIAAAAADAAAEAAAAAAAAAAAAAAAAABOq2YQN4xM3e9IsJVfPIgi/////wAAAAEAAAAAAAAAAAAAAAEAAAAGEqHCS0KXzFEqy+P7DnpM+c82rApjbQBnMGUCMBlbseoO/TU0Ap2UZ8XTO29sErz90jRdyv/k0j/i2NNXKyYrLHSpm5+hUkSokRvgAwIxAMTt3y/x4bzv5GMmNuIoZduElik538c72dEDX43z/AQc/QKjFHm6omhuBVD+sZo9UA=="
data, err1 := b64.StdEncoding.DecodeString(ciphertext_regular)
// Decrypt the data
result1, err1 := svc.Decrypt(&kms.DecryptInput{CiphertextBlob: data})
if err1 != nil {
fmt.Println("Got error decrypting data: ", err1)
os.Exit(1)
}
blob_string := string(result1.Plaintext)
fmt.Println(blob_string)
}
I've tried to look it up online, and it seems like people are doing the same thing as me. (here)
The complete error message:
Got error decrypting data: InvalidCiphertextException:
status code: 400, request id: 35f1ec73-18e3-11e8-9f54-8793fe969339
Anyone has an idea why?
Doug's comment is incorrect.
Anyways, are you setting the encryption context during encryption? If so, this will fail your decryption. Can you post some sample code of your encryption?
The Go dev guide has KMS examples of encrypting/decrypting in Go: https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/kms-example-encrypt-data.html and https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/kms-example-decrypt-blob.html, respectively.
From your posted ciphertext I can see that you encrypted your data with an encryption context (as #Xibz mentions). You need to supply this same encryption context when Decrypt'ing your data. https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/concepts.html#encryption-context
I am learning about the best practices involved in REST API design and wrote a function which handles the GET /cities HTTP/1.1 query.
This function contains cities which is a struct array that holds the cityname, citycode of multiple cities.
Below is the code
func FindCitiesHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json;charset=UTF-8")
if len(cities) == 0 {
w.WriteHeader(404)
return
}
if err := json.NewEncoder(w).Encode(cities); err != nil {
/* what to do here? */
}
w.WriteHeader(200)
}
Now when I started thinking about the possible outcomes of this function. I found these situations.
It returns all the cities properly as JSON response successfully. So I return 200 http status.
The list of cities is empty. So there is nothing to return. So I return 404 (resource not found)
It is about to return JSON response of all cities, but something gone wrong during JSON encoding. Now I am confused here, how do I deal with this situation.
I mean how do you convey message properly to user, if
If your business/application logic had some error/exception.
If data access logic found some issues. (say connection to database is not reachable)
Could you guys please help me to suggest best practices you followed in these situations?
200 is correct
404 is probably not correct; A list of cities can be empty, and still exist. (Think: A 0-length array is still an array.) You should probably return 200. You would only return 404 if the list of cities doesn't exist on your server (in other words, that should probably never happen with your API).
If you experience an internal server error, such as with marshaling JSON, you should return an Internal Server Error, status 500.
I'm using huandu/facebook for Golang to access the FB API. https://github.com/huandu/facebook
This works really well locally but when I try to run from the Google App Engine environment, I can't get it to run.
I used this code locally:
res, err := fb.Get("/me", fb.Params{
"fields": "id,first_name,last_name,name",
"access_token": usertoken,
})
In the documentation (link above) they do mention the App Engine environment but I can'f figure out how to ge this to work with the fb.Get convention.
Thanks.
Edit
Almost got it to work!:
// create a global App var to hold app id and secret.
var globalApp = fb.New("<appId>", "<appSecret>")
session := globalApp.Session(usertoken) //User token here
context := appengine.NewContext(r) //Not sure what r should be...
session.HttpClient = urlfetch.Client(context)
res, err := session.Get("/me", nil)
if err := json.NewEncoder(w).Encode(res); err != nil {
panic(err)
}
If I do this I get back the Id and name. Now all I need to do is request the other parameters. Do I do this in the r parameter to the app engine context?
To answer the last question asked, the appengine.NewContext(r) function takes a *http.Request as a parameter, but this refers to the current request, the one your code is executing in. You can use r.URL.Query() if you wanted to get the query parameters that were sent to this request.
If you want to send parameters in another request, like the one to the Facebook API, you can include them directly in the URL you pass to session.Get(). You can use url.Values.Encode if you want to build a query string from a map of values. If you need to make a request using a method other than GET, such as to an API method that expects JSON, you can use http.NewRequest eg.
session.HttpClient = urlfetch.Client(context)
request, err := http.NewRequest("PUT", url, strings.NewReader("{ "someProperty": 1234 }"))
response, err := session.Do(request)
below is the code when user uploads a video from mobile application to S3
def uploadVideo = Action(parse.multipartFormData) { implicit request =>
try {
var height = 0
var width = 0
request.body.files.map { mov =>
var videoName = System.currentTimeMillis() + ".mpeg"
amazonS3Client.putObject(bucketVideos, videoName, mov.ref.file)
}
val map = Map("result" -> "success")
Ok(write(map))
} catch {
case e: Exception =>
Ok(write(Map("result" -> "error")))
}
}
the above code work fine but in case user cancel while uploading of video then error occurs
[error] play - Exception caught in RequestBodyHandler
java.nio.channels.ClosedChannelException: null
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.cleanUpWriteBuffer(AbstractNioWorker.java:434) ~[netty.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.writeFromUserCode(AbstractNioWorker.java:129) ~[netty.jar:na]
at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSocket(NioServerSocketPipelineSink.java:99) ~[netty.jar:na]
at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:36) ~[netty.jar:na]
at org.jboss.netty.channel.Channels.write(Channels.java:725) ~[netty.jar:na]
at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:71) ~[netty.jar:na]
and this doesn't go to catch block!!
1.can this is harmfull to server or not?(because it is not needed any response if error occours)
2.if yes, how to handle?
This is all happening in Play's internals that are handling parsing the body of the Request. In fact, during the upload to your server, you haven't even reached the try block yet because the file hasn't finished uploading. Only once the upload is complete do you have the TemporaryFile available.
So no, you can't catch this error, and why would you want to? The user closed the connection. They're not even waiting for a response, so why send one? Let Play handle it.
This is also not a good way of handling an upload, though. For small files, it's passable, but if someone is proxying a huge video upload through your server to S3, it's going to:
Take almost twice is long to serve the response (which will cause the user to hang while you upload to S3).
Block one of Play's threads for handling requests for the entire time that file is uploading to S3, and given enough of these uploads (not many at all), you will no longer be able to process requests until an upload has completed.
Consider at least creating a separate ExecutionContext to use for handling uploads, or even better, look into having the user upload directly to S3 via a signed form, which would remove the need to proxy the upload at all.