OneDrive REST API - Upload - Content of .png file converted to jpeg - rest

When using the OneDrive REST API to upload files, the content of some files with a .png extension is automatically converted to JPEG. I'm able to reproduce the problem with both documented upload methods.
Sample png files:
http://www39.zippyshare.com/v/59255310/file.html
http://www2.zippyshare.com/v/11270772/file.html
For reference the shortened requests:
PUT method
PUT https://apis.live.net/v5.0/folder.<removed>/files/i2.png HTTP/1.1
Pragma: no-cache
<removed - no Content-Type header>
Content-Length: 33579
Host: apis.live.net
When the Content-Type in the header of the PUT request is set to application/octet-stream the following error is returned:
HTTP/1.1 415 Unsupported Media Type
Server: Live-API/19.7.925.4009 Microsoft-HTTPAPI/2.0
<removed>
{
"error": {
"code": "request_body_invalid_media_type",
"message": "The Content-Type header 'application/octet-stream' isn't supported."
}
}
POST method
POST https://apis.live.net/v5.0/folder.<removed>/files HTTP/1.1
Content-Type: multipart/form-data; boundary=735b1931-a2bb-4970-8142-373848528fcb
<removed>
Content-Length: 33767
Host: apis.live.net
--735b1931-a2bb-4970-8142-373848528fcb
Content-Disposition: form-data; name="file"; filename="i2.png"
Content-Type: application/octet-stream
.PNG<removed>
--735b1931-a2bb-4970-8142-373848528fcb--
Other OneDrive APIs
The problem does not happen with the API (skyapi.onedrive.live.com/API/2) Microsoft is using on the OneDrive website and in the Windows desktop client (skydrive.exe).
OneDrive developers:
Instead of returning a 415 error in the PUT request, would it be possible to support requests where the Content-Type: application/octet-stream header is set? In this case the data should be left as is without doing any data conversion. Thank you.

This is documented behaviour as per http://msdn.microsoft.com/en-us/library/dn659726.aspx (see the first note under uploading).
If you want to disable conversion, you can set downsize_photo_uploads=false in your query string.

Related

How do I POST a JSON body AND a Video file to an API endpoint on JMeter?

I'm unable to send both, JSON body data and a video/image file together in one request on JMeter. It gives a 401 unauthorized, even though I have added the Authorization header in the HTTP Header Manager. The same request with headers works fine on Postman.
I've already referred to this guide here with no luck.
https://www.blazemeter.com/blog/testing-advanced-rest-api-file-uploads-jmeter/
Content-Type: multipart/form-data
{
"email": "xyz#gmail.com",
"password": "xyz",
"file": ${__FileToString("/Downloads/SampleVideo_1280x720_1mb.mp4")}
}
SAMPLER RESULT:
Thread Name: Thread Group 1-1
Sample Start: 2019-09-30 14:26:25 IST
Load time: 2922
Connect Time: 838
Latency: 2922
Size in bytes: 202
Sent bytes:1056397
Headers size in bytes: 202
Body size in bytes: 0
Sample Count: 1
Error Count: 1
Data type ("text"|"bin"|""):
Response code: 401
Response message: Unauthorized
HTTPSampleResult fields:
ContentType:
DataEncoding: null
REQUEST:
POST data:
--o3F8APyqP080W3wk0N_-96jzl11Bfsa
Content-Disposition: form-data; name="file"; filename="SampleVideo_1280x720_1mb.mp4"
Content-Type: video/mp4
Content-Transfer-Encoding: binary
<actual file content, not shown here>
--o3F8APyqP080W3wk0N_-96jzl11Bfsa--
[no cookies]
JMeter should be doing what real browser (or other application using your API endpoint) is doing.
If the browser (or the application) sends 2 requests - JMeter must be sending 2 requests as well. If the browser (or the application) sends 1 request - JMeter must be sending 1 request. In case of 1 request my expectation is that you should be sending application/json as the value of the Content-Type header like it's done in the guide you're referring.
--boundary
Content-Type: application/json; charset=UTF-8
JSON Metadata
--boundary
Content-Type: file MIME type
File content
--boundary--
The answer is quite simple: just record the request using JMeter's HTTP(S) Test Script Recorder and JMeter will come up with proper configuration of the HTTP Request sampler(s), HTTP Header Manager, etc. All you will need to do is to implement parameterization and correlation so the script could be replayed successfully.
One important bit: during the recording you need to have SampleVideo_1280x720_1mb.mp4 file in JMeter's "bin" folder as modern browsers don't return the full path hence JMeter will be able to properly capture the file upload request only if the file lives in its "bin" folder. More information: Recording File Uploads with JMeter
To post json body and attachment you can add your "email" and "password" in Parameters tab of JMeter HTTPS request and file in Files Upload tab.

How to upload an image using AWS API Gateway Proxy Integration with S3

After setting up my API to upload files, I realised that there is a special case where you want to upload a picture (jpg), you defined the binary support at the API, but you get the following error:
The request signature we calculated does not match the signature you
provided. Check your AWS Secret Access Key and signing method.
Consult the service documentation for details.
The Canonical String for this request should have been
'PUT /test/vi-dummy-bucket/testImg2.jpg
content-type:application/x-www-form-urlencoded
host:qhweyos7z2.execute-api.us-west-1.amazonaws.com
x-amz-date:20170808T154441Z
x-amz-security-token: // security token string no quotes
content-type;host;x-amz-date;x-amz-security-token 5fa90f0 ...'
The String-to-Sign should have been
'AWS4-HMAC-SHA256\n20170808T154441Z
20170808/us-west-1/execute-api/aws4_request
f7a38fa ...'
The strange thing is that uploading simple text files works with the exact same api call, then only thing I have to change is
Content-Type 'text/plain'
and write a text in the raw portion of the request.
Not sure if this is a Content-Type issue or a Request Body Issue, if I leave everything in the working state (text/plain & text in the body) and just change the body to binary and set the image, I get the above error.
My API gateway is in us-west-1 region
My S3 bucket is in us-east-1 region
And the request I am using is:
PUT /test/vi-dummy-bucket/testImg2.jpg HTTP/1.1
Host: qhwe7z2.execute-api.us-west-1.amazonaws.com
Content-Type: application/x-www-form-urlencoded
X-Amz-Security-Token: FQoDYX ...
X-Amz-Date: 20170808T154441Z
Authorization: AWS4-HMAC-SHA256
Credential=ASIAJICO6JFTJWN7A/20170808/us-west-1/execute-
api/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-
security-token,
Signature=6a792 ... Cache-Control: no-cache
Postman-Token: e9d1f730-f50b-7e27-70cc-c15a138d8cc6
(Binary Image)
This is another version of the request (same error):
PUT /test/vi-dummy-bucket/testImg2.jpg HTTP/1.1
Content-Type: image/jpeg
x-amz-security-token: FQoDY ...
x-amz-date: 20170808T190134Z
Authorization: AWS4-HMAC-SHA256
Credential=ASIAIZSP5YKVLJ3GVVQA/20170808/us-west-1/execute-
api/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-
security-token,
Signature=b2324 ...
Host: qhos7z2.execute-api.us-west-1.amazonaws.com
Connection: close
User-Agent: Paw/3.1.2 (Macintosh; OS X/10.12.6) GCDHTTPRequest
Content-Length: 823236
--- UPDATE ---
After implementing the sigV4 sigining manually using the generated SDK, the signature is no longer an issue.
The only problem left, is that the generated SDK only accepts a string as the "body", so I have to convert the file to a binary string. Then it passes correctly and a file is created in S3, but the size is now double and its not viewable, as if the binary string wasn't converted back to the binary file. So frustrating...
BTW, I've already tried PASSTHROUGH and CONVERT_TO_BINARY.
Updated: It looks like this may be related to a known error in Postman. For reference here is a related SO question: AWS Signature Error using Postman to access the AWS API Gateway when posting a binary
and here is the bug report for Postman: https://github.com/postmanlabs/postman-app-support/issues/3232
Does the request work if you use an alternate rest client and/or a command line utility like curl or httpie?
If you configured the binary support you should probably set the Content-Type to match the binary content you're sending.
From what you've posted you're sending the binary content with Content-Type application/x-www-form-urlencoded but if the body is actually a binary jpeg file I'd expect that you should be sending Content-Type image/jpeg

Performance Center API to upload script

I'm developing a Java tool using HttpClient to programmatically upload a VuGen script to Performance Center. What my tool do are:
Authenticate and save session cookie - /LoadTest/rest/authentication-point/authenticate
Upload the script - /LoadTest/rest/domains/[MyDomain]/projects/[MyProject]/Scripts
Log out - /LoadTest/rest/authentication-point/logout
http://alm-help.saas.hpe.com/en/12.53/api_refs/Performance_Center_REST_API/Performance_Center_REST_API.htm#scripts.htm
Authenticate and Log out step are working correctly. Namely, I can get session cookie from authentication and set cookie to be expired when logging out. However, at the upload script step, I always get HTTP 401 error. This is an error message:
401 - Unauthorized: Access is denied due to invalid credentials.
You do not have permission to view this directory or page using the
credentials that you supplied.
This is what I sent to the Performance Center server.
POST http://[PC-Host]/LoadTest/rest/domains/[MyDomain]/projects/[MyProject]/Scripts HTTP/1.1
Content-Type: multipart/form-data
Content-Length: 31367
Host: [PC-Server]
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_111)
Cookie: LWSSO_COOKIE_KEY=1y8TbNawvT976BENi4oT1hW6_dCMwcz-eohdFDqWpfIXsW2tUMYHZuHZbUBg9wFQwFnQgfetx5I2EvCfaA5y-g..; QCSession=NjA3NTY5O0JWK0ZjSExzWStwTFZPd2xZSXZ5VlEqKjtQQyBSRVNUIEFQSSBDbGllbnQ7IDsg
Accept-Encoding: gzip,deflate
--_NK2aWTiwhT6VV0TIxkUIaHRx0GhDBZ2LOsY8
Content-Disposition: form-data; name="xml"
Content-Type: application/xml; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
<Script xmlns="http://www.hp.com/PC/REST/API"><TestFolderPath>Subject\DEV\abc</TestFolderPath><Overwrite>true</Overwrite><RuntimeOnly>true</RuntimeOnly><KeepCheckedOut/></Script>
--_NK2aWTiwhT6VV0TIxkUIaHRx0GhDBZ2LOsY8
Content-Disposition: form-data; name="file"; filename="abc.zip"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
## binary of zipped vugen script ##
--_NK2aWTiwhT6VV0TIxkUIaHRx0GhDBZ2LOsY8--
I have no idea what I'm doing wrong. Do you have any ideas?
Problem solved. XML encoding was to to blame.
I also moved to requests toolbelt to have more control on the multipart structure.

Azure Media Services AssetDeliveryPolicies in REST Api - do they gone?

I'm writing application in which i upload video, encode it, and then i need to expose a link to encoded video. One of the steps is to create AssetDeliveryPolicies and link it to Asset, i tried to do that, but to my surprise i don't see such entity as AssetDeliveryPolicies on that endpoint:
https://wamsamsclus001rest-hs.cloudapp.net/api/AssetDeliveryPolicies
Do anybody have tried to do something like that? I'm not sure whether i'm doing something wrong or it is already deprecated and handled in some other way.
Create an AccessDeliveryPolicy
POST https://media.windows.net/api/AssetDeliveryPolicies HTTP/1.1
Content-Type: application/json
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 3.0;NetFx
Accept: application/json
Accept-Charset: UTF-8
Authorization: Bearer [token here]
x-ms-version: 2.11
x-ms-client-request-id: [guid here]
Host: media.windows.net
{"Name":"Clear Policy",
"AssetDeliveryProtocol":7,
"AssetDeliveryPolicyType":2,
"AssetDeliveryConfiguration":null}
Link an AssetDeliveryPolicy to an Asset.
POST https://media.windows.net/api/Assets('nb%3Acid%3AUUID%3A86933344-9539-4d0c-be7d-f842458693e0')/$links/DeliveryPolicies HTTP/1.1
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 3.0;NetFx
Accept: application/json
Accept-Charset: UTF-8
Content-Type: application/json
Authorization: Bearer [token here]
x-ms-version: 2.11
x-ms-client-request-id: [new guid here]
Host: media.windows.net
{"uri":"https://media.windows.net/api/AssetDeliveryPolicies('nb%3Aadpid%3AUUID%3A92b0f6ba-3c9f-49b6-a5fa-2a8703b04ecd')"}
Where nb%3Acid%3AUUID%3A86933344-9539-4d0c-be7d-f842458693e0 is the URL Encoded value of AssetId and nb%3Aadpid%3AUUID%3A92b0f6ba-3c9f-49b6-a5fa-2a8703b04ecd is the URL Encoded value of the AccessDeliveryPolicy Id.
Source

REST Upload to Skydrive without Content-Length

I'm trying to upload a file to Skydrive where I don't a-priori know the Content-Length. With other storage services I can do this with chunked HTTP upload, but Skydrive always complains about Content-Length being invalid.
Here are the full headers I'm sending:
PUT /v5.0/me/skydrive/files/skydrive_test.js?overwrite=ChooseNewName HTTP/1.1
Authorization: Bearer <TOKEN_REDACTED>
host: apis.live.net
content-type: application/javascript
Connection: keep-alive
Transfer-Encoding: chunked
Here's the response I get back:
cache-control: private, no-cache, no-store, must-revalidate
transfer-encoding: chunked
content-type: application/json; charset=UTF-8
server: Live-API/16.4.1731.327 Microsoft-HTTPAPI/2.0
x-content-type-options: nosniff
x-http-live-request-id: API.c6afda25-2d9f-4248-9f49-001ccb3a9007
x-http-live-server: BAYMSG1010836
date: Wed, 15 May 2013 14:33:00 GMT
{ "error": { "code": "request_invalid_content_length",
"message": "The value for the Content-Length header isn't valid." }}
Is there any way I can do this without setting Content-Length (i.e. using chunked encoding)?
I'm using node.js to do this, but it should apply equally with any language using the REST API, hence I haven't tagged this with a particular language.
For example Dropbox offers the Chunked Upload command: https://www.dropbox.com/developers/core/docs#chunked-upload
And Google Drive, even though it says it wants Content-Length, doesn't need it for it's resumable upload API: https://developers.google.com/drive/manage-uploads#resumable
Is there an API I'm missing?
Edit: Things I've tried: Setting Content-Length: 0 results in it working, but the file is zero bytes. Setting Content-length:0 and Transfer-Encoding: chunked, results in the original error above.
Try setting a dummy content-length to see if it is acceptable. Otherwise set the file size.
If you are using node.js you can get the file size you are trying to upload and set the size to content-length in OCTETs. You can get the file size requiring the fs (filesystem) module in node.
var fs = require('fs');
fs.watchFile('some.file', function () {
fs.stat('some.file', function (err, stats) {
console.log(stats.size);
});
});
#Resource