Google Storage API - Retrieve object with "/" in name - google-cloud-storage

When I try to retrieve an object with a "/" in the name I get all of the file information and metadata in the response, but the link to the file itself just points to "not found". This applies to both in my code, and using the API Explorer on their website.
For example, a file named "testfolder/test.txt". It shows up in the Developer Console correctly as a pseudo-folder named "testfolder/" and a file inside named "test.txt", but when do a Get on "testfolder/test.txt", or List with prefix of "testfolder/" I run into the issue mentioned above.
I can retrieve the object file fine in gsutil and from the console. I can also retrieve the object file fine with the API if I use a symbol other than "/". It's just something about the "/" causing the link from the API response to not point to the file correctly.
Request:
GET https://www.googleapis.com/storage/v1/b/dschaffertestbucket/o/testfolder%2Ftest.txt?key={YOUR_API_KEY}
Response:
200 OK
{
"kind": "storage#object",
"id": "dschaffertestbucket/testfolder/test.txt/1429992872601000",
"selfLink": "https://www.googleapis.com/storage/v1/b/dschaffertestbucket/o/testfolder%2Ftest.txt",
"name": "testfolder/test.txt",
"bucket": "dschaffertestbucket",
"generation": "1429992872601000",
"metageneration": "1",
"contentType": "application/octet-stream",
"updated": "2015-04-25T20:14:32.600Z",
"storageClass": "STANDARD",
"size": "14",
"md5Hash": "vLrQ6JkgmdfYKJKKN1ebFQ==",
"mediaLink": "https://www.googleapis.com/download/storage/v1/b/dschaffertestbucket/o/testfolder%2Ftest.txt?generation=1429992872601000&alt=media",
"owner": {
"entity": "user-00b4903a971fb6d48e9e442442b7892dfc55f81ce026106122cb58989d926f00",
"entityId": "00b4903a971fb6d48e9e442442b7892dfc55f81ce026106122cb58989d926f00"
},
"crc32c": "Qcfj8Q==",
"etag": "CKjLz8miksUCEAE="
}

In this article you can find that Cloud Storage doesn't have a file hierarchy like Windows and Linux, and that 'gsutil' and Cloud Storage APIs simply give the illusion of one.
In your case it would be easier to just avoid including the "/" character in your file names, or use a different character instead.

Related

Seeks 'commentsResolvedState' API proposal, but it doesn't exist

[warning] Via 'product.json#extensionEnabledApiProposals' extension
'github.vscode-pull-request-github' wants API proposal
'commentsResolvedState' but that proposal DOES NOT EXIST. Likely, the
proposal has been finalized (check 'vscode.d.ts') or was abandoned.
I don't know what is this error msgs mean is it a bug on the current version of vscode 1.75.1?
I don't know why it's called extension even though I don't install any extension named github.vscode-pull-request-github
also I tried to (check 'vscode.d.ts') in my laptop. I dont find any file of it, there's only vscode.d. Well My code and project still works find but this error msg always appear every time I opened my vs code and kind of annoying to see. I use windows 11
This is what I found on github:
https://github.com/microsoft/vscode-pull-request-github/pull/4447/commits/f36acaff7b81f077db18e74a7c673cf249eba996
I tried to put the code in setting.json but it seems doesn't work. this is the code:
{
"name": "vscode-pull-request-github",
"displayName": "%displayName%",
"description": "%description%",
"icon": "resources/icons/github_logo.png",
"repository": {
"type": "git",
"url": "https://github.com/Microsoft/vscode-pull-request-github"
},
"bugs": {
"url": "https://github.com/Microsoft/vscode-pull-request-github/issues"
},
"enabledApiProposals": [
"tokenInformation",
"contribShareMenu",
"treeItemCheckbox",
"contribCommentPeekContext",

Unable to upload the talend job zip file from windows machine to tac through metaservlet api

Unable to call the associatePreGeneratedJob api endpoint. Keep getting the 180 error. Trying to upload the zip file from Windows machine to a tac server that is running on docker container
Talend documentation:
Command: associatePreGeneratedJob
Description : Create a new execution task with a pre-generated zip file
- targetConductor: could be [JOBCONDUCTOR, BIGDATA_STREAMING]. "JOBCONDUCTOR" is the default value.
- importType: the position where select the zip file from; 'Nexus' or 'File', default value is 'File'
Requires authentication : true
Since : 5.2
Sample :
{
"actionName": "associatePreGeneratedJob",
"active": true,
"authPass": "admin",
"authUser": "admin#company.com",
"contextName": "Default",
"description": "task1's description",
"executionServerName": "serv1",
"filePath": "'/home/talend/generatedJob.zip'",
"importType": "Nexus",
"logLevel": "Info",
"nexusArtifactId": "test",
"nexusGroupId": "org.example",
"nexusRepository": "snapshots",
"nexusVersion": "0.1.0-SNAPSHOT",
"onUnknownStateJob": "WAIT",
"pauseOnError": false,
"taskName": "task1",
"timeout": 3600
}
Specific error codes :
180: file is not a valid file or not exist
181: nexus parameters may be not correct
Reference:
Talend Metaservlet api documentation: https://help.talend.com/reader/oYf9gKhmYrkWCiSua4qLeg/SLiAyHyDTjuznLR_F~MiQQ
Have tried this.
MetaServletCaller.bat --tac-url=http://localhost:8080/org.talend.administrator/ --json-params="{"actionName":"associatePreGeneratedJob","active": true,"authPass": "admin","authUser": "admin#company.com","contextName": "Default","description":"New","executionServerName": "test", "filePath": "'C:\Users\st\Desktop\JOb\t1.zip'","logLevel": "Info","onUnknownStateJob": "WAIT","importType": "File","pauseOnError": false, "taskName": "task_new","taskType": "Normal","timeout": 3600}"
Job zip file should be uploaded to tac server
Going to the network logs and capturing the api request helped to decipher where the files were getting stored in the container. If zip files are somehow sent to that location in the container, that path can be passed to the api request
Ex path in container : /usr/local/tomcat/temp/test.zip

How to upload file to Google drive with Postman using multipart?

How could I upload file to Google Drive using Google Drive API with Postman?
I also want to set file's name and parent directory.
What have I done:
I try to upload file with following properties:
But I get 403 error:
{
"error": {
"errors": [
{
"domain": "global",
"reason": "parseError",
"message": "Parse Error"
}
],
"code": 400,
"message": "Parse Error"
}
}
I have resolved my problem, using answer on this question: How to send application/json data along with file in postman multipart/form-data post request?.
Solution
1) create file for upload to Google Drive: test.txt;
2) create file upload-options.json with file's metadata properties in json format:
{
"name": "my-uploaded-file.txt",
"parents": ["<parent-directory-id>"]
}
3) add two form-data fields:
first - upload-options.json
second - test.txt
you should consider about permission scope of your app.
add line code below help you access full control of google driver api
const SCOPES = ['https://www.googleapis.com/auth/drive'];

Why using Google Cloud Drive Rest API file.list can not get all the files?

I am using the following CURL command to retrieve all my google drive files, however, it only list a very limited part of the whole bunch of files. Why?
curl -H "Authorization: Bearer ya29.hereshouldbethemaskedaccesstokenvalue" https://www.googleapis.com/drive/v3/files
result
{
"kind": "drive#fileList",
"incompleteSearch": false,
"files": [
{
"kind": "drive#file",
id": "2fileidxxxxxxxx",
"name": "testnum",
"mimeType": "application/vnd.google-apps.folder"
},
{
"kind": "drive#file",
"id": "1fileidxxxxxxx",
"name": "test2.txt",
...
}
token scope includes
https://www.googleapis.com/auth/drive.file
https://www.googleapis.com/auth/drive.appdata
Using the Android SDK also facing the same issue.
Any help would be appreciated.
Results from files.list are paginated -- your response should include a "nextPageToken" field, and you'll have to make another call for the next page of results. See documentation here about the files list call. You may want to use one of the client libraries to make this call (see the examples at the bottom of the page)
I have the same problem when try to get list of files in Google Drive folder. This folder has more than 5000 files, but API return only two of them. The problem is -- when files in folder shared with anyone with a link, in fact it isn't shared with you until you open it. Owner of this folder must specify you as viewer.

Upload a file to my github repo from postman

I would like to upload a file to my Github repo from Postman. What I have tried is:
Generate token from PAT: https://github.com/settings/tokens.
Method PUT URL: https://github.com/username/test2/info/lfs/objects/cd00e292c5970d3c5e2f0ffa5171e555bc46bfc4faddfb4a418b6840b86e79a2
Body is a 1 MB file.
I am receiving the following error: Your browser did something unexpected. Please contact us if the problem persists.
According to the API Reference you can't simply PUT a file to that URL. Rather, you need to encode the file as Base64, and put it within a JSON object with the following inputs:
{
"message": "my commit message",
"committer": {
"name": "Scott Chacon",
"email": "schacon#gmail.com"
},
"content": "bXkgbmV3IGZpbGUgY29udGVudHM="
}