first, i have read, searched google and everywhere. but i cant find the solution to my problem. I can upload images to my google cloud storage bucket. then, i want to get the public URL, so i can save it to my mongodb, and access it later on for my site.
I have a code here, written in GO.
// profile func, that receive filename, and file
name, self_link, err := google_storage.Upload(handler.Filename, file)
// then after acquiring the self_link, i update the user
db.UpdateUser(session_id.(string), db.UserUpdate{"ProfileImage": self_link})
the problem is, i can't access the file from my link to my pages when passing it to the html..
i guess, my links aren't public?
here is my oauth config
config := &oauth.Config{
ClientId: clientId,
ClientSecret: clientSecret,
Scope: storage.DevstorageFull_controlScope,
AuthURL: "https://accounts.google.com/o/oauth2/auth",
TokenURL: "https://accounts.google.com/o/oauth2/token",
TokenCache: oauth.CacheFile(cacheFile),
RedirectURL: "https://www.example.com/oauth2callback",
AccessType: "offline",
}
from there after authorizing my web server, i can just
// my object setup
object_to_store_data := &storage.Object{Name: fileName}
result, err := service.Objects.Insert(bucketName, storage_data).Media(file).Do()
any help is appreciated, im new to google cloud storage, and go.
thank you in advance
Related
This page:
https://console.cloud.google.com/iam-admin/serviceaccounts
if you go into an individual project that has a service account, and use the three dots on that row to execute the action "Get Key", it will generate a json that looks like:
{
"type": "service_account",
"project_id": "project name",
"private_key_id": "sdfljsdlfsdljkf",
"private_key": "-----BEGIN PRIVATE KEY-----\nsldjflsdjfkljsdf\n-----END PRIVATE KEY-----\n",
"client_email": "user#projectname.iam.gserviceaccount.com",
"client_id": "9797979797998",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/projectname.iam.gserviceaccount.com"
}
This doesn't match with what is needed for google.auth.OAuth2(client_id, client_secret, redirect_uris[])
What do I do to create a token to use with google spreadsheets api that uses this key information. It seems that the redirect_uris and the client secret are missing.
Update: the goal of this is to send google calendar events as other users under our domain.
I believe your goal and situation as follows.
You want to use Sheets API using the service account.
You want to achieve this using googleapis for Node.js.
You have already enabled Sheets API at API console.
In this case, google.auth.JWT is used instead of google.auth.OAuth2(). A simple sample script for this is as follows.
Sample script:
Before you use this script, please set the variable for google.auth.JWT(). In this case, as a samplet, client_email and private_key are set. And please set Spreadsheet ID. And, in this case, please share your sample Google Spreadsheet in your Google Drive with the email of the service account. By this, the service account can read the Spreadsheet.
const { google } = require("googleapis");
const spreadsheetId = "###"; // Please set Spreadsheet ID.
const auth = new google.auth.JWT(
"###", // Please set client_email here.
null,
"-----BEGIN PRIVATE KEY-----\n###\n-----END PRIVATE KEY-----\n", // Please set private_key here,
["https://www.googleapis.com/auth/spreadsheets.readonly"],
null
);
const sheets = google.sheets({ version: "v4", auth: auth });
sheets.spreadsheets
.get({ spreadsheetId: spreadsheetId })
.then(({ data }) => console.log(data))
.catch((err) => console.log(err));
Note:
When you run above script and an error like The caller does not have permission occurs, please confirm that whether the Spreadsheet of spreadsheetId is shared with the email of the service account again. If it is not, please share it with the email of the service account.
Please confirm whether the version of googleapis is the latest one.
References:
googleapis for Node.js
Method: spreadsheets.get
I'm trying to get information of a file that is inside of a folder in Google cloud Storage (not firebase storage). But the API of it in Dart language is not complete, it doesn't have a function to show the blob (file) information like we have in python same API. I just need to access the name of the file. Here's my code:
var credentials = auth.ServiceAccountCredentials.fromJson({
"type": "service_account",
...
});
List<String> scopes = []..addAll(Storage.SCOPES);
var client = await auth.clientViaServiceAccount(credentials, scopes);
var storage = Storage(client, "project_name");
var bucket = storage.bucket("Bucket_name");
var list = await bucket.read("folder_name/");
list.forEach((element) {
print(element.toString());
});
It has a lot of options like toList(),toSet(), asBroadcastStream() and etc. But any of these return me what I need. Some ones just return a empty list, that doesn't make sence for me.
Anyways, if someone know how to read data from a folder of GCP storage, please anwser me. Sorry for my english and Thanks!
The API docs: https://pub.dev/documentation/gcloud/latest/gcloud.storage/gcloud.storage-library.html
For you backend, you can use bucket.list(prefix: "folder_name/","") as described in the documentation:
Listing operates like a directory listing, despite the object namespace being flat. Unless delimiter is specified, the character / is being used to separate object names into directory components. To list objects recursively, the delimiter can be set to empty string.
For the front end, forget this! You can't provide a service account key file in your frontend! If you share the secret publicly, it's like if you set your bucket public!
So, for this, you need to have a backend that authenticated the user and generate a signed URL to read and write inside the bucket.
bucket.read() gets the contents of individual objects. If you want to get the object names, use bucket.list().
I am getting this error when trying to create a "transfer" to transfer the contents of one bucket in Google Cloud to another bucket in Google Cloud under the same owner:
To complete this transfer, you need the 'storage.buckets.setIamPolicy' permission for the source bucket. Ask the bucket's administrator to grant you the required permission and try again.
I have no idea what I'm supposed to do. I tried going to "Bucket -> Permissions -> Add Members -> myemail.com for Storage -> ...Admin" but I just keep getting "IAM policy update failed".
Please help on what to do to get this working so I can make my files publicly accessible.
I am using Node.js if that helps.
If I even try to fetch the photo and bypass it directly, I can't even do that :/
const { Storage } = require('#google-cloud/storage')
const storage = new Storage({
projectId: 'my-bucket'
})
const bucket = storage.bucket('my.bucket')
app.get('/photo/:photo.:ext', (req, res) => {
const remoteFile = bucket.file(`photo/${req.params.photo}.${req.params.ext}`)
remoteFile.createReadStream().pipe(res)
})
Can't do this either:
const opts = {
includeFiles: true
};
bucket.makePublic(opts, function(err, files) {
// `err`:
// The first error to occur, otherwise null.
//
// `files`:
// Array of files successfully made public in the bucket.
console.log(arguments)
});
Cannot get legacy ACLs for a bucket that has enabled Bucket Policy Only. Read more at https://cloud.google.com/storage/docs/bucket-policy-only.
$ gsutil iam ch allUsers:objectViewer gs://my.bucket
ServiceException: 401 Anonymous caller does not have storage.buckets.getIamPolicy access to my.bucket.
The error clearly indicates a missing permission on the Source bucket. I recommend you confirm that the Owner on the Source bucket has the Permission, storage.objects.getIamPolicy(IAM&admin --> IAM Menu --> Filter by the Owner's email address --> check the role on it). Then, you can check if the roles has that permission, storage.objects.getIamPolicy (go to IAM&admin -->Roles and then, search for specific role --> Click on it and it would show the list of assigned permission. Ensure that storage.objects.getIamPolicy is one of the permissions listed for the Role
Meanwhile, for you to be able to grant access to specific buckets, your account role must be a Storage Admin. So, if your account does not have that role, you would need someone that has that role to be able to grant access or have other control over the bucket
I expect you have found a solution to this on your own.
But for anyone getting the same error message i got this when trying to set up a transfer from the cloud console. I had done a transfer before between these two storage buckets and not changed the name of the transfer, so the second time trying this the suggested name was the same as the previous. I changed the transfer name to a unique one and that solved the issue for me.
No idea why i kept getting this error message as permissions was not part of the problem.
I'm trying to use postman to do REST API calls to firebase. I've managed to read from firebase when my security rule is to permit all users including unauthorized ones.
but when I use this rule :
{"rules":{".read": "auth != null", ".write": "auth != null"}}
I get 'error' : 'permission denied' from postman.
I did the request token for google's web oauth2.0 client and got the authorization_code token back.
I tried to use token in the URL and in the header, tried it with GET & POST request and still get denied.
please help.
Thanks in advance
The answers above did not work for me.
What did work for me was going to
Project Settings (top left corner gear) -> Service Accounts (far right tab) -> Database Secrets (left menu) -> Scroll down, hover over the bulltets and click Show
Use this as the auth key, i.e. .../mycollection.json?auth=HERE
For me it worked like this:
https://your-database-url/users.json?auth=YOUR_AUTH_KEY
Where can you get this AUTH_KEY?
you get this key from your Project Settings -> Database -> Secret Key
Try something like this
https://your-database-url/users.json?auth=YOUR_AUTH_KEY
Respone is a JSON of your USERS node
I created a Postman pre-request script for helping create a Authentication: Bearer JWT. Should save a lot of copy pasting when testing APIs with Firebase Auth. https://gist.github.com/moneal/af2d988a770c3957df11e3360af62635
Copy of script at time of posting:
/**
* This script expects the global variables 'refresh_token' and 'firebase_api_key' to be set. 'firebase_api_key' can be found
* in the Firebase console under project settings then 'Web API Key'.
* 'refresh_token' as to be gathered from watching the network requests to https://securetoken.googleapis.com/v1/token from
* your Firebase app, look for the formdata values
*
* If all the data is found it makes a request to get a new token and sets a 'auth_jwt' environment variable and updates the
* global 'refresh_token'.
*
* Requests that need authentication should have a header with a key of 'Authentication' and value of '{{auth_jwt}}'
*
* Currently the nested assertions silently fail, I don't know why.
*/
pm.expect(pm.globals.has('refresh_token')).to.be.true;
pm.expect(pm.globals.has('firebase_api_key')).to.be.true;
var sdk = require('postman-collection'),
tokenRequest = new sdk.Request({
url: 'https://securetoken.googleapis.com/v1/token',
method: 'POST',
body: {
mode: 'urlencoded',
urlencoded: [{
type: 'text',
key: 'key',
value: pm.globals.get('firebase_api_key')
},
{
type: 'text',
key: 'grant_type',
value: 'refresh_token'
},
{
type: 'text',
key: 'refresh_token',
value: pm.globals.get('refresh_token')
},
]
}
});
pm.sendRequest(tokenRequest, function(err, response) {
pm.test('request for access token was ok', function() {
pm.expect(response).to.be.ok();
});
const json = response.json();
pm.expect(json).to.an('object');
pm.test('response json has needed properties', function() {
pm.expect(json).to.have.own.property('access_token');
pm.expect(json).to.have.own.property('token_type');
pm.expect(json).to.have.own.property('refresh_token');
const accessToken = json.access_token;
const tokenType = json.token_type;
const refreshToken = json.refresh_token;
pm.environment.set('auth_jwt', tokenType + ' ' + accessToken);
pm.globals.set('refresh_token', refreshToken);
});
});
Note: Adding this answer as all Options listed here is either deprecated or not working(mostly due to missing steps).
Best way to make it work with Postman is to use Google OAuth2 access tokens. The provided link described in full length but I have added quick steps.
Step 1: Download Service-Accounts.json
Step 2: Generate Access token in Java (provided link described support in other language for this)
make sure to include this dependency:
implementation 'com.google.api-client:google-api-client:1.25.0'
OR
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client</artifactId>
<version>1.25.0</version>
</dependency>
Run this code to generate token(Copied from google's javadocs)
// Load the service account key JSON file
FileInputStream serviceAccount = new FileInputStream("path/to/serviceAccountKey.json");
GoogleCredential scoped = GoogleCredential
.fromStream(serviceAccount)
.createScoped(
Arrays.asList(
"https://www.googleapis.com/auth/firebase.database",
"https://www.googleapis.com/auth/userinfo.email"
)
);
// Use the Google credential to generate an access token
scoped.refreshToken();
String token = scoped.getAccessToken();
System.out.println(token);
Step 3: Use the token in Postman
It is very simple to fetch data via Postman:
Here is how I did it
1 Your DB URL
https://YOUR_PROJECT_URL.firebaseio.com/YOUR_STRUCTURE/CLASS.json
2 Add API key in header as auth
auth = Value of your API_KEY
example:
We can use Firebase with postman for rest APIs
How we use Firebase in postman?
Copy your Firebase database URL
Paste it into your postman URL
Add .json at the last like shown in the image
Then play with your firebase database and make REST API
I'm using huandu/facebook for Golang to access the FB API. https://github.com/huandu/facebook
This works really well locally but when I try to run from the Google App Engine environment, I can't get it to run.
I used this code locally:
res, err := fb.Get("/me", fb.Params{
"fields": "id,first_name,last_name,name",
"access_token": usertoken,
})
In the documentation (link above) they do mention the App Engine environment but I can'f figure out how to ge this to work with the fb.Get convention.
Thanks.
Edit
Almost got it to work!:
// create a global App var to hold app id and secret.
var globalApp = fb.New("<appId>", "<appSecret>")
session := globalApp.Session(usertoken) //User token here
context := appengine.NewContext(r) //Not sure what r should be...
session.HttpClient = urlfetch.Client(context)
res, err := session.Get("/me", nil)
if err := json.NewEncoder(w).Encode(res); err != nil {
panic(err)
}
If I do this I get back the Id and name. Now all I need to do is request the other parameters. Do I do this in the r parameter to the app engine context?
To answer the last question asked, the appengine.NewContext(r) function takes a *http.Request as a parameter, but this refers to the current request, the one your code is executing in. You can use r.URL.Query() if you wanted to get the query parameters that were sent to this request.
If you want to send parameters in another request, like the one to the Facebook API, you can include them directly in the URL you pass to session.Get(). You can use url.Values.Encode if you want to build a query string from a map of values. If you need to make a request using a method other than GET, such as to an API method that expects JSON, you can use http.NewRequest eg.
session.HttpClient = urlfetch.Client(context)
request, err := http.NewRequest("PUT", url, strings.NewReader("{ "someProperty": 1234 }"))
response, err := session.Do(request)