I'm developing a Java tool using HttpClient to programmatically upload a VuGen script to Performance Center. What my tool do are:
Authenticate and save session cookie - /LoadTest/rest/authentication-point/authenticate
Upload the script - /LoadTest/rest/domains/[MyDomain]/projects/[MyProject]/Scripts
Log out - /LoadTest/rest/authentication-point/logout
http://alm-help.saas.hpe.com/en/12.53/api_refs/Performance_Center_REST_API/Performance_Center_REST_API.htm#scripts.htm
Authenticate and Log out step are working correctly. Namely, I can get session cookie from authentication and set cookie to be expired when logging out. However, at the upload script step, I always get HTTP 401 error. This is an error message:
401 - Unauthorized: Access is denied due to invalid credentials.
You do not have permission to view this directory or page using the
credentials that you supplied.
This is what I sent to the Performance Center server.
POST http://[PC-Host]/LoadTest/rest/domains/[MyDomain]/projects/[MyProject]/Scripts HTTP/1.1
Content-Type: multipart/form-data
Content-Length: 31367
Host: [PC-Server]
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_111)
Cookie: LWSSO_COOKIE_KEY=1y8TbNawvT976BENi4oT1hW6_dCMwcz-eohdFDqWpfIXsW2tUMYHZuHZbUBg9wFQwFnQgfetx5I2EvCfaA5y-g..; QCSession=NjA3NTY5O0JWK0ZjSExzWStwTFZPd2xZSXZ5VlEqKjtQQyBSRVNUIEFQSSBDbGllbnQ7IDsg
Accept-Encoding: gzip,deflate
--_NK2aWTiwhT6VV0TIxkUIaHRx0GhDBZ2LOsY8
Content-Disposition: form-data; name="xml"
Content-Type: application/xml; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
<Script xmlns="http://www.hp.com/PC/REST/API"><TestFolderPath>Subject\DEV\abc</TestFolderPath><Overwrite>true</Overwrite><RuntimeOnly>true</RuntimeOnly><KeepCheckedOut/></Script>
--_NK2aWTiwhT6VV0TIxkUIaHRx0GhDBZ2LOsY8
Content-Disposition: form-data; name="file"; filename="abc.zip"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
## binary of zipped vugen script ##
--_NK2aWTiwhT6VV0TIxkUIaHRx0GhDBZ2LOsY8--
I have no idea what I'm doing wrong. Do you have any ideas?
Problem solved. XML encoding was to to blame.
I also moved to requests toolbelt to have more control on the multipart structure.
Related
Axis2 by default is send a multipart response even when there are no attachments
Why is axis2 sending a multipart response & how to ask it to send a "application/xml" or "application/soap+xml" how to get rid of multipart?
In the axis2.xml I have
In the response I see
HTTP/1.1 200 OK
Date: Fri, 17 Feb 2017 01:07:08 GMT
Transfer-Encoding: chunked
Content-Type: multipart/related; boundary="MIMEBoundary_87162747c87b279f7caa4e1ab573d5d864a878de7fae1a0b"; type="application/xop+xml"; start="<0.97162747c87b279f7caa4e1ab573d5d864a878de7fae1a0b#apache.org>"; start-info="text/xml"
--MIMEBoundary_87162747c87b279f7caa4e1ab573d5d864a878de7fae1a0b
Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"
Content-Transfer-Encoding: binary
Content-ID: <0.97162747c87b279f7caa4e1ab573d5d864a878de7fae1a0b#apache.org>
200<?xml version="1.0" encoding="UTF-8"?>
<List_Wrapper>
<_bp>
<_comments><_comment><content>Again a new comment this is a text type bp comment need to see this in the text json 1</content><published_date>2017-01-18T21:07:15</published_date><published_by>cyril furtado</published_by><company>Chevron Inc.</company></_comment></_comments></_bp>
</List_Wrapper>
--MIMEBoundary_87162747c87b279f7caa4e1ab573d5d864a878de7fae1a0b-
I too faced similar issue for AXIS 2 quite recently , so thought to answer this.
AXIS 2 by default support attachment processing globally, which might not be needed for all types of services. To resolve this issue I disabled MTOM processing globally by modifying axis2.xml file -
<parameter name="enableMTOM">false</parameter>
This capability can now be enabled on need basis per service basis by enabling this property via respective services.xml file
I've got a simple class making a HTTP POST request (to a Neo4j server, but that shouldn't matter) using URLSession, and an integration test that tests that it succeeds.
Through Xcode, via the iOS simulator, I can run this test a million times - it succeeds every time.
However, when I run swift test on Ubuntu Linux (I use the docker image provided by IBM) then I very often will get a test failure saying I got a 401 response.
Replicating that via cURL in the container does not fail at all, so I do not believe this to be a problem with my container.
I have added a packet dump (that I inspect via Charles Proxy) where our of 15 test runs, 7 test runs (and thus 7 requests) fail. All of the failed requests complain that no authentication header was supplied. And from the dump, that is correct, the dump does not contain the authentication header for those requests that fail. But why not? In fact, all of the header flags are different: a successful run has these headers:
POST /db/data/cypher HTTP/1.1
Host: 192.168.0.18:7474
Accept-Encoding: deflate, gzip
Authorization: Basic bmVvNGo6c3RhY2swdmVyRmxvdw==
Content-Type: application/json; charset=utf-8
Accept: application/json; charset=utf-8
Connection: keep-alive
User-Agent: urlsessionTestPackageTests.xctest (unknown version) curl/7.35.0
Content-Length: 135
while an unsuccessful one has these:
POST /db/data/cypher HTTP/1.1
Host: 192.168.0.18:7474
Accept: */*
Accept-Encoding: deflate, gzip
Connection: keep-alive
User-Agent: urlsessionTestPackageTests.xctest (unknown version) curl/7.35.0
Content-Length: 135
All the 200 results have the same headers, and all the 401 results have the same headers. Can you see anything in my code that should warrant such a random request?
When using the OneDrive REST API to upload files, the content of some files with a .png extension is automatically converted to JPEG. I'm able to reproduce the problem with both documented upload methods.
Sample png files:
http://www39.zippyshare.com/v/59255310/file.html
http://www2.zippyshare.com/v/11270772/file.html
For reference the shortened requests:
PUT method
PUT https://apis.live.net/v5.0/folder.<removed>/files/i2.png HTTP/1.1
Pragma: no-cache
<removed - no Content-Type header>
Content-Length: 33579
Host: apis.live.net
When the Content-Type in the header of the PUT request is set to application/octet-stream the following error is returned:
HTTP/1.1 415 Unsupported Media Type
Server: Live-API/19.7.925.4009 Microsoft-HTTPAPI/2.0
<removed>
{
"error": {
"code": "request_body_invalid_media_type",
"message": "The Content-Type header 'application/octet-stream' isn't supported."
}
}
POST method
POST https://apis.live.net/v5.0/folder.<removed>/files HTTP/1.1
Content-Type: multipart/form-data; boundary=735b1931-a2bb-4970-8142-373848528fcb
<removed>
Content-Length: 33767
Host: apis.live.net
--735b1931-a2bb-4970-8142-373848528fcb
Content-Disposition: form-data; name="file"; filename="i2.png"
Content-Type: application/octet-stream
.PNG<removed>
--735b1931-a2bb-4970-8142-373848528fcb--
Other OneDrive APIs
The problem does not happen with the API (skyapi.onedrive.live.com/API/2) Microsoft is using on the OneDrive website and in the Windows desktop client (skydrive.exe).
OneDrive developers:
Instead of returning a 415 error in the PUT request, would it be possible to support requests where the Content-Type: application/octet-stream header is set? In this case the data should be left as is without doing any data conversion. Thank you.
This is documented behaviour as per http://msdn.microsoft.com/en-us/library/dn659726.aspx (see the first note under uploading).
If you want to disable conversion, you can set downsize_photo_uploads=false in your query string.
I'm trying to upload a file to Skydrive where I don't a-priori know the Content-Length. With other storage services I can do this with chunked HTTP upload, but Skydrive always complains about Content-Length being invalid.
Here are the full headers I'm sending:
PUT /v5.0/me/skydrive/files/skydrive_test.js?overwrite=ChooseNewName HTTP/1.1
Authorization: Bearer <TOKEN_REDACTED>
host: apis.live.net
content-type: application/javascript
Connection: keep-alive
Transfer-Encoding: chunked
Here's the response I get back:
cache-control: private, no-cache, no-store, must-revalidate
transfer-encoding: chunked
content-type: application/json; charset=UTF-8
server: Live-API/16.4.1731.327 Microsoft-HTTPAPI/2.0
x-content-type-options: nosniff
x-http-live-request-id: API.c6afda25-2d9f-4248-9f49-001ccb3a9007
x-http-live-server: BAYMSG1010836
date: Wed, 15 May 2013 14:33:00 GMT
{ "error": { "code": "request_invalid_content_length",
"message": "The value for the Content-Length header isn't valid." }}
Is there any way I can do this without setting Content-Length (i.e. using chunked encoding)?
I'm using node.js to do this, but it should apply equally with any language using the REST API, hence I haven't tagged this with a particular language.
For example Dropbox offers the Chunked Upload command: https://www.dropbox.com/developers/core/docs#chunked-upload
And Google Drive, even though it says it wants Content-Length, doesn't need it for it's resumable upload API: https://developers.google.com/drive/manage-uploads#resumable
Is there an API I'm missing?
Edit: Things I've tried: Setting Content-Length: 0 results in it working, but the file is zero bytes. Setting Content-length:0 and Transfer-Encoding: chunked, results in the original error above.
Try setting a dummy content-length to see if it is acceptable. Otherwise set the file size.
If you are using node.js you can get the file size you are trying to upload and set the size to content-length in OCTETs. You can get the file size requiring the fs (filesystem) module in node.
var fs = require('fs');
fs.watchFile('some.file', function () {
fs.stat('some.file', function (err, stats) {
console.log(stats.size);
});
});
#Resource
I'm trying to implement file upload functionality in the iPhone app. Server code is tested and works when files are uploaded from the desktop browser, so I moved to implementing the Objective-C client code. I'm assembling HTTP requests body manually, and despite that it looks correct, it is rejected by the server (server handler unable to extract the parts from multipart content). In desperation I've simplified the form to having only one parameter, but it still does not work.
I've captured the network traffic and I could see that Wireshark could not parse my multipart content as well (have a look at screenshots: Firefox request, iPhone request). I'm pasting it below in hope that you could see the errors I can't see.
Thanks in advance.
Firefox:
POST /cubepaint/actions/gallery/post HTTP/1.1
Host: [...]
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-GB; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Authorization: Basic [...]
Content-Type: multipart/form-data; boundary=---------------------------20072377098235644401115438165
Content-Length: 180
-----------------------------20072377098235644401115438165
Content-Disposition: form-data; name="deviceId"
12345
-----------------------------20072377098235644401115438165--
HTTP/1.1 200 OK
Date: Sat, 17 Oct 2009 22:09:21 GMT
Server: Apache/2.2.3 (Debian) DAV/2 SVN/1.4.2 mod_python/3.2.10 Python/2.4.4 mod_ssl/2.2.3 OpenSSL/0.9.8c
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
iPhone:
POST /cubepaint/actions/gallery/post HTTP/1.1
Host: [...]
User-Agent: Copenhagen/1.0 CFNetwork/459 Darwin/9.8.0
Content-Type: multipart/form-data; boundary=----------0E7B16E6-CD3D-4213-9B42-07DA30822C74
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Authorization: Basic [...]
Content-Length: 187
Connection: keep-alive
----------0E7B16E6-CD3D-4213-9B42-07DA30822C74
Content-Disposition: form-data; name="deviceId"
00000000-0000-1000-8000-0016CBCC0B61
----------0E7B16E6-CD3D-4213-9B42-07DA30822C74--
HTTP/1.1 200 OK
Date: Sat, 17 Oct 2009 22:04:07 GMT
Server: Apache/2.2.3 (Debian) DAV/2 SVN/1.4.2 mod_python/3.2.10 Python/2.4.4 mod_ssl/2.2.3 OpenSSL/0.9.8c
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
Your iPhone version indicates keep-alive but doesn't specify a length. Not sure that's enough to cause trouble.
Also, is it possible your server is checking for user-agent strings it recognizes (say, for backward-compatibility mode)?
I'd also compare the two in a text editor that shows CR/LF characters to make sure you're getting proper line endings.
Another thing you could try is create a simple web-page that does a multipart POST and run it from the iPhone browser (instead of the Mac one) then check the headers that go across the wire. Or you could snag a toolkit like ASIHTTPRequest and see what kind of output it generates for multi-part posts (or just use the toolkit instead of trying to write your own).
Good luck
Solved by reading RFC 2046 (MIME specification): boundary between parts of multipart message should contain two leading '-'s, and last boundary should additionally contain two trailing '-'s. The boundary in the request header and request body in the Firefox request differ:
---------------------------20072377098235644401115438165
and
-----------------------------20072377098235644401115438165
The last boundary looks like this:
-----------------------------20072377098235644401115438165--
You really could not see this with the eye when there are so many leading '-'s in the original boundary.