I'm new to Golang, and am stuck on this issue for a day already.
I am trying to write a small code snippet that decrypts (using AWS go sdk) the cipher text which was previously encrypted (using AWS python sdk). I don't see any problem doing this since the encrypted value is just a base64 encoded string, but I keep running into this error, InvalidCiphertextException, when trying to decrypt.
Here's my GO code:
package main
import (
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/kms"
"fmt"
"os"
"github.com/aws/aws-sdk-go/aws"
b64 "encoding/base64"
)
func main() {
// Initialize a session in us-west-2 that the SDK will use to load
// credentials from the shared credentials file ~/.aws/credentials.
sess, err := session.NewSession(&aws.Config{
Region: aws.String("us-east-1")},
)
fmt.Println(err)
// Create KMS service client
svc := kms.New(sess)
ciphertext_regular := "AYADeGDoLfFyXv59h9zNJFR+QcgAXwABABVhd3MtY3J5cHRvLXB1YmxpYy1rZXkAREF4ZFRKNjhiZmM3eTNoUTR3S2JsVjNwMHprNFh6NnVrUVdXRUxvMTFuVEpPRnZLODhTMm5CWUdERm85WU5XeGVaZz09AAEAB2F3cy1rbXMAS2Fybjphd3M6a21zOnVzLWVhc3QtMTo0ODIyMzQ4MTEwMDQ6a2V5L2Y0ZWFhOWZjLThlMTYtNGFkNi1iMjVkLWE2NzM3Y2E2NDg4ZQC4AQIBAHiqDEdiAJHv6/rKQOUfXvXVzNboRyJDQ7oz+NqStcjb1AGvjNtfT8oLIQ9Ju7i5TD0HAAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMhZXqEceZ0lRyt8ehAgEQgDuhJmDyZ9RJS6+lPy7qSarpaXC7v1MVV0EH3r5XVpwbhTMy0/f2Clyu/OMeSfib/n5Kdkm9AjtSDLaiDgIAAAAADAAAEAAAAAAAAAAAAAAAAABOq2YQN4xM3e9IsJVfPIgi/////wAAAAEAAAAAAAAAAAAAAAEAAAAGEqHCS0KXzFEqy+P7DnpM+c82rApjbQBnMGUCMBlbseoO/TU0Ap2UZ8XTO29sErz90jRdyv/k0j/i2NNXKyYrLHSpm5+hUkSokRvgAwIxAMTt3y/x4bzv5GMmNuIoZduElik538c72dEDX43z/AQc/QKjFHm6omhuBVD+sZo9UA=="
data, err1 := b64.StdEncoding.DecodeString(ciphertext_regular)
// Decrypt the data
result1, err1 := svc.Decrypt(&kms.DecryptInput{CiphertextBlob: data})
if err1 != nil {
fmt.Println("Got error decrypting data: ", err1)
os.Exit(1)
}
blob_string := string(result1.Plaintext)
fmt.Println(blob_string)
}
I've tried to look it up online, and it seems like people are doing the same thing as me. (here)
The complete error message:
Got error decrypting data: InvalidCiphertextException:
status code: 400, request id: 35f1ec73-18e3-11e8-9f54-8793fe969339
Anyone has an idea why?
Doug's comment is incorrect.
Anyways, are you setting the encryption context during encryption? If so, this will fail your decryption. Can you post some sample code of your encryption?
The Go dev guide has KMS examples of encrypting/decrypting in Go: https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/kms-example-encrypt-data.html and https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/kms-example-decrypt-blob.html, respectively.
From your posted ciphertext I can see that you encrypted your data with an encryption context (as #Xibz mentions). You need to supply this same encryption context when Decrypt'ing your data. https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/concepts.html#encryption-context
Related
I am trying to create a JWK server in a pod (on k8s) which would serve the keys when a request for a particular kid comes in. I create the JWK using the generate function
func (s *jwkServer) startJWKServer() {
privateKey, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
panic(err)
}
jwk := go_jose.JSONWebKey{
Key: privateKey,
KeyID: "rand_key_id",
Algorithm: "RSA256",
}
s.jwk = &jwk
}
func (s *jwkServer) ServeHTTP(response http.ResponseWriter, request *http.Request) {
// Do some encoding and decoding to encapsulate into the response object
return s.jwk
}
I used the above generated key to sign the JWT token.
When the request for the above stated key comes in, the pod returns the key.
The problem here is that if the pod restarts before the request for the JWK comes in, then the key cannnot be found and this results in an error. How should such a scenario be handled ?
I thought about a couple of solutions here
Solution 1: Create the same key every time but then this would be a problem if someone gets access to the key.
Solution 2: A new JWK key is generated everytime the pod restarts but then I am concerned about the case where the pod restart happens a lot of times and then it adds latency to the code.
How should I handle the JWK server in kubernetes ?
Here is how we solved it
Public key is persisted somewhere ( either DB or a PV or a vault)
When the request comes in and the pod has not restarted, we would have the public key in memory, the server returns the key
When the request comes in and the pod HAS restarted, we wont have the public key in memory. But we would still have it in our persistence layer, we would serve the public key from the persistence layer.
answered
I am having a hard time with Mongodb and Gridfs, using it with Go's http package. I am trying to store a .mp4 file into Gridfs, then pull it out into the browser for playback.
Heres what I am doing now. It successfully pulls the file from
database, I could even write it correctly to a download location.
// Connect to database
// Session to database
func movie(w http.ResponseWriter r *http.Request) {
file, err := db.GridFS("fs").Open("movie.mp4")
if err != nil {
log.Println(err)
}
defer file.Close()
w.Header.Set("Content-type", "video/mp4")
if _, err := io.Copy(w, file); err != nil {
log.Println(err)
}
// I am trying to send it to the browser.
// I want to achieve same thing as, http://localhost/view/movie.mp4,
as if you did that.
}
If the file was on the server, I would just do something like this. But instead I am trying to store it in Mongodb, for all around easier use involving metadata.
func movie(w http.ResponseWriter r *http.Request) {
http.ServeFile("./uploads/movie.mp4") // Easy
}
The browser is receiving something, but its just malformed or corrupted. Just shows the video player with an error message. Any help would be appreciated, I have only been programming a week.
Here is a picture of the error, no console error messages.
Unless someone has an alternative to storing video files for playback
in browser somewhere other than MongoDB or Amazon S3. Please let me know,
thanks.
You may want to check http.ServeContent. It will handle all the mess (content-type, content-length, partial data, cache) automaticlly and save you a lot of time. It takes a ReadSeeker to serve, which GridFile already implented. So your code may simply change to the below.
func movie(w http.ResponseWriter r *http.Request) {
file, err := db.GridFS("fs").Open("movie.mp4")
if err != nil {
log.Println(err)
}
defer file.Close()
http.ServeContent(w,r,"movie.mp4",file.UploadDate(),file)
}
If that doesn't work, please use tools like curl or wget to download the served content and compare it to the orignal one (in db).
I'm using huandu/facebook for Golang to access the FB API. https://github.com/huandu/facebook
This works really well locally but when I try to run from the Google App Engine environment, I can't get it to run.
I used this code locally:
res, err := fb.Get("/me", fb.Params{
"fields": "id,first_name,last_name,name",
"access_token": usertoken,
})
In the documentation (link above) they do mention the App Engine environment but I can'f figure out how to ge this to work with the fb.Get convention.
Thanks.
Edit
Almost got it to work!:
// create a global App var to hold app id and secret.
var globalApp = fb.New("<appId>", "<appSecret>")
session := globalApp.Session(usertoken) //User token here
context := appengine.NewContext(r) //Not sure what r should be...
session.HttpClient = urlfetch.Client(context)
res, err := session.Get("/me", nil)
if err := json.NewEncoder(w).Encode(res); err != nil {
panic(err)
}
If I do this I get back the Id and name. Now all I need to do is request the other parameters. Do I do this in the r parameter to the app engine context?
To answer the last question asked, the appengine.NewContext(r) function takes a *http.Request as a parameter, but this refers to the current request, the one your code is executing in. You can use r.URL.Query() if you wanted to get the query parameters that were sent to this request.
If you want to send parameters in another request, like the one to the Facebook API, you can include them directly in the URL you pass to session.Get(). You can use url.Values.Encode if you want to build a query string from a map of values. If you need to make a request using a method other than GET, such as to an API method that expects JSON, you can use http.NewRequest eg.
session.HttpClient = urlfetch.Client(context)
request, err := http.NewRequest("PUT", url, strings.NewReader("{ "someProperty": 1234 }"))
response, err := session.Do(request)
I have build coturn and run it successfully. ip:192.168.1.111. Now the question I faced is to get the Turn credential through REST API.
https://datatracker.ietf.org/doc/html/draft-uberti-behave-turn-rest-00 According to the passage the request format should be
GET /?service=turn&username=mbzrxpgjys
and response should be JSON. Now my question is:
a) How to configure and command TURN SERVER to make it run in REST API mode?
b) How to write a http request in the right format so TURN SERVER can reply correctly? could you give me an example?
Few things to be clarified here are:
GET /?service=turn&username=mbzrxpgjys which returns a JSON, is just a suggested uri for retrieving time-limited TURN credentials from the server, you do not have to follow that, your uri can be just /?giveMeCredentials. In fact, I use my socket connection to retrieve this data, not direct http call with json response. End of day, it does not matter how you( the client that uses said TURN) get those credentials as long as they are valid.
You do not make any requests to the TURN server directly, no rest api call to TURN server is under your control.
you allocate a secret key when you are starting the TURN server, this can be taken from a db(thus dynamically changable), but lazy that I am, just hard-coded, and gave it in the turn config file, also remember to enable REST API. As part of turn command, turnserver ... --use-auth-secret --static-auth-secret=MySecretKey
Now, in your application server, you would use the same secret key to generate credentials, for username, it is UNIX timestamp and some string( can be random or user id or something) seperated by : and the password would be HMAC of the username with your secret key.
about the UNIX timestamp, this has be the time in TURN server till which your credentials has to be valid, so which calculating this make sure you take into account of the clock time difference between your application server and your turn server.
Now some sample code taken from my answer to another question
command for stating TURN server:
turnserver -v --syslog -a -L xx.xxx.xx.xx -X yy.yyy.yyy.yy -E zz.zzz.zz.zzz --max-bps=3000000 -f -m 3 --min-port=32355 --max-port=65535 --use-auth-secret --static-auth-secret=my_secret --realm=north.gov --cert=turn_server_cert.pem --pkey=turn_server_pkey.pem --log-file=stdout -q 100 -Q 300 --cipher-list=ALL
node.js code for creating TURN credentials in application server:
var crypto = require('crypto');
function getTURNCredentials(name, secret){
var unixTimeStamp = parseInt(Date.now()/1000) + 24*3600, // this credential would be valid for the next 24 hours
username = [unixTimeStamp, name].join(':'),
password,
hmac = crypto.createHmac('sha1', secret);
hmac.setEncoding('base64');
hmac.write(username);
hmac.end();
password = hmac.read();
return {
username: username,
password: password
};
}
Browser code for using this:
...
iceServers:[
{
urls: "turn:turn_server_ip",
username: username,
credential:password
}
...
After (many) hours of frustration, #Mido's excellent answer here was the only thing that actually got CoTurn's REST API working for me.
My credential server is PHP and I use CoTurn's config file 'turnserver.conf' so here's a tested and working translation of Mido's work for that situation:
Assuming a 'shared secret' of '3575819665154b268af59efedee8826e', here are the relevant turnserver.conf entries:
lt-cred-mech
use-auth-secret
static-auth-secret=3575819665154b268af59efedee8826e
...and the PHP (which misled me for ages):
$ttl = 24 * 3600; // Time to live
$time = time() + $ttl;
$username = $time . ':' . $user;
$password = base64_encode(hash_hmac('sha1', $username, '3575819665154b268af59efedee8826e', true));
Building upon #Mido and #HeyHeyJC answers, here is the Python implementation to build credentials for coturn.
import hashlib
import hmac
import base64
from time import time
user = 'your-arbitrary-username'
secret = 'this-is-the-secret-configured-for-coturn-server'
ttl = 24 * 3600 # Time to live
timestamp = int(time()) + ttl
username = str(timestamp) + ':' + user
dig = hmac.new(secret.encode(), username.encode(), hashlib.sha1).digest()
password = base64.b64encode(dig).decode()
print('username: %s' % username)
print('password: %s' % password)
Here is a web application to test the login to your coturn server. Use turn:host.example.com as the server name.
I came across similar issue (getting REST API working with TURN server) recently and learned that TURN server doesn't support REST API calls at all and just provides support for an authentication format with shared secret when we enable REST API support in TURN config. The draft only provides info on things that we need to consider while implementing such REST API and WE need to create the API on our own or use something like turnhttp to generate the temporary username password combo.
As #mido detailed, you can implement the username/password generation part in the application itself. But if you have reasons to separate this from the application and want to implement it as an entirely different API service, instead of implementing a complete API as per the draft, I came across another post in which the OP provided a PHP script to generate temp username & password and this one works pretty well once you modify the hash_hmac() function to the following,
$turn_password = hash_hmac('sha1', $turn_user, $secret_key, true);
We need to base64 encode the RAW output of hash_hmac to get it working and I believe this is why it was not working for the OP in that link.
You should be able to test authentication using turnutils_uclient command to verify that the temp username/password combo is working as expected.
turnutils_uclient -y -u GENERATED_USERNAME -w GENERATED_PASSWORD yourturnserver.com
Once you have verified authentication and confirmed that it's working, you can setup webserver for the PHP script to make it available to your application and fetch the temporary username/password combo. Also, you would need to implement other security setup (authentication) to protect the API from unauthorized access.
I know this is an old post, just sharing my findings here hoping that it will be useful for someone someday.
Here is my c# implementation with TTL
public string[] GenerateTurnPassword(string username)
{
long ttl = 3600 * 6;
var time = DateTimeOffset.Now.ToUnixTimeSeconds() + ttl;
var newuser = time + ":" + username;
byte[] key = Encoding.UTF8.GetBytes("YOURSECRET");
HMACSHA1 hmacsha1 = new HMACSHA1(key);
byte[] buffer = Encoding.UTF8.GetBytes(newuser);
MemoryStream stream = new MemoryStream(buffer);
var hashValue = hmacsha1.ComputeHash(stream);
string[] arr = new string[2];
arr[0] = Convert.ToBase64String(hashValue);
arr[1] = newuser;
return arr;
}
Well #Augusto Destrero provided implementation will cause TypeError: key: expected bytes or bytearray, but got 'str' on Python 3.7.6, for anyone looking for another Python implementation, here is an example:
import time
import hmac
import hashlib
import base64
secret = b'abcdefghijkmln'
def generateTurnUsernamePwd():
username = "arbitry username here"
password = hmac.new(secret, bytes(username, 'UTF-8'), hashlib.sha1).digest()
passwordStr = base64.b64encode(password).decode("utf-8")
return username,passwordStr
print(generateTurnUsernamePwd())
The main difference is key and message keyword arguments in hmac lib has to be bytes in newer version , while in older versions, it requires str.
I thought it worthwhile to add to the answer the actual text of the documentation of coturn regardingg this topic and a link to it for those interested:
--auth-secret TURN REST API flag. Flag that sets a special WebRTC authorization option that is based upon authentication secret. The
feature purpose is to support "TURN Server REST API" as described
in the TURN REST API section below. This option uses timestamp
as part of combined username: usercombo -> "timestamp:username",
turn user -> usercombo, turn password ->
base64(hmac(input_buffer = usercombo, key = shared-secret)). This
allows TURN credentials to be accounted for a specific user id. If
you don't have a suitable id, the timestamp alone can be used. This
option is just turns on secret-based authentication. The actual
value of the secret is defined either by option static-auth-secret,
or can be found in the turn_secret table in the database.
Here is an example for go with ttl:
import (
"crypto/hmac"
"crypto/sha1"
"encoding/base64"
"fmt"
"time"
)
const turnTokenTtl = time.Hour * 24
const turnSecret = "your secret"
func getTurnCredentials(name string) (string, string) {
timestamp := time.Now().Add(turnTokenTtl).Unix()
username := fmt.Sprintf("%d:%s", timestamp, name)
h := hmac.New(sha1.New, []byte(turnSecret))
h.Write([]byte(username))
credential := base64.StdEncoding.EncodeToString(h.Sum(nil))
return username, credential
}
first, i have read, searched google and everywhere. but i cant find the solution to my problem. I can upload images to my google cloud storage bucket. then, i want to get the public URL, so i can save it to my mongodb, and access it later on for my site.
I have a code here, written in GO.
// profile func, that receive filename, and file
name, self_link, err := google_storage.Upload(handler.Filename, file)
// then after acquiring the self_link, i update the user
db.UpdateUser(session_id.(string), db.UserUpdate{"ProfileImage": self_link})
the problem is, i can't access the file from my link to my pages when passing it to the html..
i guess, my links aren't public?
here is my oauth config
config := &oauth.Config{
ClientId: clientId,
ClientSecret: clientSecret,
Scope: storage.DevstorageFull_controlScope,
AuthURL: "https://accounts.google.com/o/oauth2/auth",
TokenURL: "https://accounts.google.com/o/oauth2/token",
TokenCache: oauth.CacheFile(cacheFile),
RedirectURL: "https://www.example.com/oauth2callback",
AccessType: "offline",
}
from there after authorizing my web server, i can just
// my object setup
object_to_store_data := &storage.Object{Name: fileName}
result, err := service.Objects.Insert(bucketName, storage_data).Media(file).Do()
any help is appreciated, im new to google cloud storage, and go.
thank you in advance