I have generated my KMS key in Golang using this code
// Create the key
result, err := svc.CreateKey(&kms.CreateKeyInput{
Tags: []*kms.Tag{
{
TagKey: aws.String("CreatedBy"),
TagValue: aws.String("User"),
},
},
})
And this is the line causing the error during Decrypt
//Get body bytes
body, _ := ioutil.ReadAll(resp.Body)
//base64Text := make([]byte, base64.StdEncoding.EncodedLen(len(body)))
//base64.StdEncoding.Encode(base64Text, []byte(body))
//Needs decrypting?
var newfile *bytes.Reader
if decrypt == true {
log.Println("Now decrypting data")
log.Println("resp.SSECustomerAlgorithm : ", resp.SSECustomerAlgorithm)
log.Println("resp.SSEKMSKeyId : ", aws.StringValue(resp.SSEKMSKeyId)) //right key found.
newkms := inputfs.NewKms()
//n := map[string]*string{"CreatedBy": aws.String("User") }
params := &kms.DecryptInput{
CiphertextBlob: body,
//EncryptionContext:n,
}
output, err := newkms.Decrypt(params)
if err != nil {
log.Println("Decrypt error :", err)
}
newbody := output.Plaintext
newfile = bytes.NewReader(newbody)
}
Error at Decrypt error
Decrypt error : InvalidCiphertextException:
status code: 400, request id: b69a8634-1784-4c57-8d3d-2439041249fe
What could possibly cause this? Nothing I do seems to work. Here is a related question: Decrypting using AWS go sdk but i don't see the answer.
It turns out my objects were already decrypted.
My objects were originally KMS encrypted using S3 PUTobject operation.
AWS S3 automatically decrypts such objects on S3 GETobject operation.
I can actually get the particular object in plaintext, so this supports the above statement.
See Request Headers and the Amazon S3 API
Related
apple will send a Consumption_Request notify to merchant server if user start a refund.
When I tried to process the Consumption Request notify and call the Send Consumption Info interface to send user consumption info to apple server, I got two forms of response.
case 1 :
when my param is wrong , the response like this :enter image description here
case 2 :
when my param is right , the response code is 401 ,like this : enter image description here
And I found 401 mean Unauthorized . So maybe my JWT check failed.
SO my question is how do you generate your JWT token? Is any demo ?
and how do you get your secret key file(the .p8 file. I suspect that I got a wrong file)?
how do you generate your JWT token?
here is one sample by golang
func readPrivateKeyFromFile(keyFile string) (*ecdsa.PrivateKey, error) {
bytes, err := ioutil.ReadFile(keyFile)
if err != nil {
return nil, err
}
block, _ := pem.Decode(bytes)
if block == nil {
return nil, errors.New("appstore private key must be a valid .p8 PEM file")
}
key, err := x509.ParsePKCS8PrivateKey(block.Bytes)
if err != nil {
return nil, err
}
switch pk := key.(type) {
case *ecdsa.PrivateKey:
return pk, nil
default:
return nil, errors.New("appstore private key must be of type ecdsa.PrivateKey")
}
}
func generateToken(privateKey *ecdsa.PrivateKey) (string, error) {
token := jwt.NewWithClaims(jwt.SigningMethodES256, jwt.MapClaims{
"iss": "issuerId,
"isa": time.Now().Unix(),
"exp": expireTIme,
"aud": "appstoreconnect-v1",
"nonce": "uuid",
"bid": "bid",
})
token.Header["kid"] = "keyId"
token.Header["alg"] = "ES256"
token.Header["typ"] = "JWT"
return token.SignedString(privateKey)
}
how do you get your secret key file(the .p8 file. I suspect that I got a wrong file)?
Please refer to doc https://developer.apple.com/documentation/appstoreserverapi/creating_api_keys_to_use_with_the_app_store_server_api?changes=latest_major
I am using crypto in go to take a password and use a passphrase to encrypt the password and then store it as a string in a postgres sql database. The encryption works fine but when I try to add it to my database I get an error that seems to indicate that going from a []byte to a string type messes up the encrypted password.
func Encrypt(password string, passphrase string) string {
data := []byte(password)
block, _ := aes.NewCipher([]byte(createHash(passphrase)))
gcm, err := cipher.NewGCM(block)
if err != nil {
panic(err.Error())
}
nonce := make([]byte, gcm.NonceSize())
if _, err = io.ReadFull(rand.Reader, nonce); err != nil {
panic(err.Error())
}
ciphertext := gcm.Seal(nonce, nonce, data, nil)
return string(ciphertext)
}
func createHash(key string) string {
hasher := md5.New()
hasher.Write([]byte(key))
return hex.EncodeToString(hasher.Sum(nil))
}
When I run this code and try to add the row in the database I get the error
ERROR #22021 invalid byte sequence for encoding "UTF8"
I am just looking for an easy way in go to encrypt a string password and save the encrypted string into a table in postgres. The column in the table is of type VARCHAR but I am willing to change that as well if that is for some reason the issue. Thanks for your time!
Base64 encode it:
return base64.StdEncoding.EncodeToString(ciphertext)
Even though a string is a byte array, not all sequences of bytes are a valid UTF-8 string. Base64 encoding is commonly used to store binary data as text.
I'm trying to store encoded data using encoding/gob from Golang into Postgres. I'm using Gorm as well.
First, sending form data using
if err := json.NewDecoder(r.Body).Decode(model); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
Currently client_encoding is set to UTF8 in the postgres database. Here's what I'm using to encode:
type payload struct {
Key []byte
Nonce *[nonceLength]byte
Message []byte
}
// Encrypt message
p.Message = secretbox.Seal(p.Message, plaintext, p.Nonce, key) // key set prior to this
buf := &bytes.Buffer{}
if err := gob.NewEncoder(buf).Encode(p); err != nil {
return nil, err
}
return buf.Bytes(), nil
Then I store string(buf.Bytes()) which is stored in the database column which is currently a string type. Now I'm a novice with encoding, and I think gob just has a different encoding for my database. I'm receiving this error in console:
(pq: invalid byte sequence for encoding "UTF8": 0xff)
I've been following this gist for encryption/decryption:
https://gist.github.com/fuzzyami/f3a7231037166117a6fef9607960aee7
From what I've read, I shouldn't be encoding structs into the db, in this case p, unless using gob. Correct me if I'm wrong with that (can't find the resource at the moment where I found this).
Is anyone able to point me in the right direction for storing this data in Postgres which is decrypted later? I'm clearly not understanding the encoding process, and not entirely sure where to start out here with reading resources, so any help is appreciated!
Use a bytea column in postgresql to store a []byte, and skip the conversion to string.
Took a look at https://golang.org/src/encoding/base64/example_test.go
Was able to use
return base64.StdEncoding.EncodeToString(buf.Bytes()), nil
Which successfully stored in the database.
I need to implement web service in go that processes tar.gz files and I wonder what is the correct way, what content type I need to define, etc.
plus, I found that a lot of things are handled automatically - on the client side I just post a gzip reader as request body and Accept-Encoding: gzip header is added automatically, and on the server side - I do not need to gunzip the request body, it is already extracted to tar. does that make sense?
Can I rely that it would be like this with any client?
Server:
func main() {
router := mux.NewRouter().StrictSlash(true)
router.Handle("/results", dataupload.NewUploadHandler()).Methods("POST")
log.Fatal(http.ListenAndServe(*address, router))
}
Uploader:
package dataupload
import (
"errors"
log "github.com/Sirupsen/logrus"
"io"
"net/http"
)
// UploadHandler responds to /results http request, which is the result-service rest API for uploading results
type UploadHandler struct {
uploader Uploader
}
// NewUploadHandler creates UploadHandler instance
func NewUploadHandler() *UploadHandler {
return &UploadHandler{
uploader: TarUploader{},
}
}
func (uh UploadHandler) ServeHTTP(writer http.ResponseWriter, request *http.Request) {
retStatus := http.StatusOK
body, err := getBody(request)
if err != nil {
retStatus = http.StatusBadRequest
log.Error("Error fetching request body. ", err)
} else {
_, err := uh.uploader.Upload(body)
}
writer.WriteHeader(retStatus)
}
func getBody(request *http.Request) (io.ReadCloser, error) {
requestBody := request.Body
if requestBody == nil {
return nil, errors.New("Empty request body")
}
var err error
// this part is commented out since somehow the body is already gunzipped - no need to extract it.
/*if strings.Contains(request.Header.Get("Accept-Encoding"), "gzip") {
requestBody, err = gzip.NewReader(requestBody)
}*/
return requestBody, err
}
Client
func main() {
f, err := os.Open("test.tar.gz")
if err != nil {
log.Fatalf("error openning file %s", err)
}
defer f.Close()
client := new(http.Client)
reader, err := gzip.NewReader(f)
if err != nil {
log.Fatalf("error gzip file %s", err)
}
request, err := http.NewRequest("POST", "http://localhost:8080/results", reader)
_, err = client.Do(request)
if err != nil {
log.Fatalf("error uploading file %s", err)
}
}
The code you've written for the client is just sending the tarfile directly because of this code:
reader, err := gzip.NewReader(f)
...
request, err := http.NewRequest("POST", "http://localhost:8080/results", reader)
If you sent the .tar.gz file content directly, then you would need to gunzip it on the server. E.g.:
request, err := http.NewRequest(..., f)
I think that's closer to the behavior you should expect third-party clients to exhibit.
Claerly not, but maybe...
Golang provides a very good support for the http client (and server). This is one of the first language to support http2 and the design of the API clearly shows their concern on having a fast http.
This is why they add Accept-Econding: gzip automatically. That will dramatically reduce the size of the server response and then optimize the transfer.
But the gzip remains an option in http 1 and not all of the client will push this header to your server.
Note that the Content-Type describes the type of data you are sending (here a tar.gz but could be application/json, test/javascript, ...), when the Accept-Encoding describes the way the data has been encoded for the transport
Go will take care of transparently handling the Accept-Encoding for you because it is responsible of the transport of the data. Then it will be up to you to handle the Content-Type because only you know how to give a sense to the content you received
Oke, I'm currently trying to login in to my school website, with my own Crawler. Altough they have some protection against login. First I do a Get request to the Website so I get the token from the hidden Input field. That token I use in my next Post request to login to the url! But for some reason the http response is that I cannot resubmit the form. But with doing the same in Postman rest client (chrome plugin) I can login!
When I try to submit a form to this url:
postLoginUrl = "?username=%s&password=%s&submit=inloggen&_eventId=submit&credentialsType=ldap<=%s"
loginUrl = "https://login.hro.nl/v1/login"
where %s are filled in credentials
req, err := client.Post(loginUrl, "application/x-www-form-urlencoded", strings.NewReader(uri))
I'm getting as response that the Form cannot be resubmitted.
But when I try it with Postman rest client, I'm allowed to login.
code for Csrf token:
func getCSRFtoken() (key string) {
doc, err := goquery.NewDocument(loginUrl)
if err != nil {
log.Fatal(err)
}
types := doc.Find("input")
for node := range types.Nodes {
singlething := types.Eq(node)
hidden_input, _ := singlething.Attr("type")
if hidden_input == "hidden" {
key, _ := singlething.Attr("value")
return key
}
}
return ""
}
goquery.NewDocument is a http.Get()
My question now is, how does the URL get's formatted from the library
Maybe you would be better off using:
(c *Client)PostForm(url string, data url.Values) (resp *Response, err error)
from net/http like http://play.golang.org/p/8D6XI6arkz
With the params in url.Values (instead of concatenating the strings, like you are doing now.)