InvalidXmlRequest error for Azure REST API request - rest

I am using the Azure REST API to create an Azure storage account using the documentation at: http://msdn.microsoft.com/en-us/library/hh264518.aspx
I keep getting the 400 error with code InvalidXmlRequest ("The request body's XML was invalid or not correctly specified."). The only related thread seemed to be at Management API - The request body XML was invalid or not correctly specified - I have tried several variations on my request (like removing xml header, removing empty elements from body, etc.) but still see the same error.
There is no requestId in the response header either (to get more info using GET OperationStatus).
The complete RAW request and response (for one of my trials) is below.
Any ideas on what I am missing here?
Request:
POST https://management.core.windows.net/<mysubscriptionid>/services/storageservices HTTP/1.1
x-ms-version: 2011-06-01
Content-Type: application/xml
Host: management.core.windows.net
Content-Length: 350
Expect: 100-continue
<?xml version="1.0" encoding="utf-8"?><CreateStorageServiceInput xmlns="http://schemas.microsoft.com/windowsAzure"><ServiceName>gjhgkjhgkgk</ServiceName><Description /><Label>gjhgkjhgkgk</Label><AffinityGroup /><Location>North Central US</Location><GeoReplicationEnabled>true</GeoReplicationEnabled><ExtendedProperties /></CreateStorageServiceInput>
Response:
HTTP/1.1 400 Bad Request
Content-Length: 228
Content-Type: application/xml; charset=utf-8
Server: Microsoft-HTTPAPI/2.0
Date: Sun, 20 Oct 2013 02:33:08 GMT
<Error xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance"><Code>InvalidXmlRequest</Code><Message>The request body's XML was invalid or not correctly specified.</Message></Error>

2 things I noticed:
Value in label element should be base64 encoded as mentioned in the documentation here: http://msdn.microsoft.com/en-us/library/windowsazure/hh264518.aspx.
Label Required. A name for the storage account specified as a
base64-encoded string. The name may be up to 100 characters in length.
The name can be used identify the storage account for your tracking
purposes.
Not related to your problem per se but you're trying to create a storage account in North Central US region. Please note that you can't create new resources in North Central and South Central US region.

Related

talend tRESTClient suspected issues with 204 No Content response without "content-length: 0" header

We use talend to build ETLs. One of the components we use is tRESTClient, to interact to a REST service in our platform. We use talend version 7.1.
Recently this REST service has changed the version of its HTTP server library (libmicrohttpd, the REST service is based in C/C++). In particular, has moved from limicrohttpd 0.9.48 to 0.9.70. And since we did the change, the integration in the ETL has started to fail. We see message likes this in the talend traces:
--------------------------------------
time=2020-08-05 08:26:01| lvl=INFO | ver=1.0| Inbound Message
----------------------------
ID: 1
Response-Code: 204
Encoding: ISO-8859-1
Content-Type:
Headers: {Cache-Control=[no-cache, no-store, max-age=0, must-revalidate], connection=[keep-alive], Date=[Wed, 05 Aug 2020 06:26:03 GMT], Expires=[0], fiware-correlator=[894f7ff0-d6e4-11ea-b1bd-fa163ec74401], Pragma=[no-cache], Referrer-Policy=[same-origin], Server=[nginx], Strict-Transport-Security=[max-age=31536000; includeSubdomains;], X-Content-Type-Options=[nosniff], X-Frame-Options=[SAMEORIGIN], X-XSS-Protection=[1; mode=block]}
--------------------------------------
Exception in component tRESTClient_1 (VLCI_ETL_MEDIOAMBIENTE_INSERT_CB)
javax.ws.rs.client.ResponseProcessingException: Problem with reading the data, class org.dom4j.Document, ContentType: */*.
at org.apache.cxf.jaxrs.impl.ResponseImpl.reportMessageHandlerProblem(ResponseImpl.java:437)
at org.apache.cxf.jaxrs.impl.ResponseImpl.doReadEntity(ResponseImpl.java:379)
at org.apache.cxf.jaxrs.impl.ResponseImpl.readEntity(ResponseImpl.java:314)
at org.apache.cxf.jaxrs.impl.ResponseImpl.readEntity(ResponseImpl.java:304)
…
We are still investigating the problem but we suspect that one of the changes introduced in libmicrohttpd could be related. In particular, with libmicrohttd 0.9.48 (when the ETL works) “Content-Length: 0” header was included in 204 responses. With libmicrohttpd 0.9.70 (when ETL doesn’t work) such “Content-Length: 0” header is not included.
Is this a known issue? I mean, is tRESTClient (version 7.1) expected to fail if “Content-Lengh: 0” is not included in 204 responses?
Note that the HTTP specification is pretty clear regarding Content-Length usage in 204 responses. From RFC 7321 (which as far as I know is the normative standard on HTTP) section 6.1:
A server MUST NOT send a Content-Length header field in any response with a status code of 1xx (Informational) or 204 (No Content)”
so if tRESTClient is not supporting it, then I understand is a bug in talend (maybe solved in post-7.1 releases?)
Any feedback on this is really welcomed! Thank you for your attention.
Finally, it seems we have solved the issue in the following way.
In the failing situation the component was using this configuration:
We have changed Accept Type from "JSON" to "Any" in the following way:
And now it is working.
My guess is that with Accept Type "JSON" the component is waiting for a response payload and this is problematic in the case of 204 No Content. Using "Any" removes that restriction.
Maybe it worked by chance with the non-standard content-length: 0 header in this case... It's a bit weird anyway, I think the tRESTClient should deal with this situation better than this. Maybe is a behavior already fixed in the post-7.1 versions.
Any feedback or comment on this solution would be welcomed. Thanks!

REST API Design: Respond with 406 or 404 if a resource is not available in a requested representation

We have a REST API to fetch binary files from the server.
The requests look like
GET /documents/e62dd3f6-18b0-4661-92c6-51c7258f9550 HTTP/1.1
Accept: application/octet-stream
For every response indicating an error, we'd like to give a reason in JSON.
The problem is now, that as the response is not of the same content type as the client requested.
But what kind of response should the server produce?
Currently, it responds with a
HTTP / 1.1 406 Not Acceptable
Content-Type: application/json
{
reason: "blabla"
...
}
Which seems wrong to me, as the underlying issue is, that the resource is not existing and not the client requesting the wrong content type.
But the question is, what would be the right way to deal with such situations?
Is it ok, to respond with 404 + application/json although application/octet-stream was requested
Is it ok, to respond with 406 + application/json, as the client did not specify an application/json as an acceptable type
Should spec been extended so that the client should use the q-param - for example, application/octet-stream, application/json;q=0.1
Other options?
If no representation can be found for the requested resource (because it doesn't exist or because the server wishes to "hide" its existence), the server should return 404.
If the client requests a particular representation in the Accept header and the server is not available to provide such representation, the server could either:
Return 406 along with a list of the available representations. (see note** below)
Simply ignore the Accept header and return a default representation of the resource.
See the following quote from the RFC 7231, the document the defines the content and semantics of the HTTP/1.1 protocol:
A request without any Accept header field implies that the user agent will accept any media type in response. If the header field is present in a request and none of the available representations for the response have a media type that is listed as acceptable, the origin server can either honor the header field by sending a 406 (Not Acceptable) response or disregard the header field by treating the response as if it is not subject to content negotiation.
Mozilla also recommends the following regarding 406:
In practice, this error is very rarely used. Instead of responding using this error code, which would be cryptic for the end user and difficult to fix, servers ignore the relevant header and serve an actual page to the user. It is assumed that even if the user won't be completely happy, they will prefer this to an error code.
** Regarding the list of available representations, see this answer.

Validate the Response for the Correct Status code inside the Response Body

I need a help on validating or fetching the status code from the response body.
The below response I received from the server.
INBOUND_MESSAGE:
HTTP/1.1 200 OK Server:Apache-Coyote/1.1 Content-Type:application/json
Content-Length:253 Date:Wed, 16 Aug 2017 17:34:21 GMT
{"statusCode":201,"message":"Product added successfully with name :
StudentGuide","result":{"id":"5994821d1aee56c4dad4f04a","name":"StudentGuide","description":"StudentsGuide","price":123.0,"forSale":true,"category":"Books"},"errors":null,"others":null}
Is there a way to validate the "statusCode" inside the response body?
I tried all the possible way but didn't succeed.
Citrus provides various methods for message validation, as described in the Citrus Documentation.
Using JSONPath validation, validation could look like this (using Java DSL):
receive(yourEndpoint)
.messageType(MessageType.JSON)
.validate("$.statusCode", "201");

CA LISA unable to create VS from Req/Rsp pairs

I have been trying to create a REST/Json Virtual service on CA LISA 7.5 (we can’t update), using request response pairs. The request response looks like the following:
Name-req
GET /cods_party_web/party/111700 HTTP/1.1
Pragma: no-cache
Cache-Control: no-cache
x-abc-outlet-id: 017879
x-abc-user-id: CTM
x-abc-consent-level: 2
x-abc-application-id: 00028
x-abc-outlet-id-type: OU_ID
x-abc-user-id-type: 1
x-IBM-Client-Id: XXX....
x-IBM-Client-Secret: XXX...
Name-rsp
HTTP/1.1 200 {"party":{"partyId":111700,"foreNames":["Julie","Pamela",""],"lastName":"Duncan","initials":["J"],"...lots of content......."type":"EMAIL"}],"associatedOU":null}
When I try to build the virtual service image, no matter what options I select, my VS image response is either in hex as shown below or it is blank.
I remember having this problem 1 year ago, and was able to get the reponse to look like below but I can’t remember how I did it.
Success response
Not found response.
Many thanks in advance
It's not returning hex - those are just column numbers for an empty binary response. I think the problem is your response document is not properly formed HTTP - there's no reason phrase in the status line, and you need two line feeds after the status line. Try this:
HTTP/1.1 200 OK
{"party":{"partyId":111700,"foreNames":["Julie","Pamela",""],"lastName":"Duncan","initials":["J"],"...lots of content......."type":"EMAIL"}],"associatedOU":null}
I understand that you can't upgrade, so this doesn't really help you, but LISA 9.5 doesn't have this issue -- the response looks like it's supposed to.
On the other hand, CA has released a free, simpler version of LISA that also successfully generates a VS from your example. Check it out here:
http://educationcontent.ca.com/A01/index.html

Is S3 REST for PUT (direct upload) operation chunked?

In S3 REST API, how does the PUT operation i.e. a direct upload not the multipart upload exactly send requests for such large files i.e. Gigabytes through HTTP? Is the direct upload also chunked (like the multipart upload) and has a defined size internally?
When tried doing a PUT (direct upload) operation using S3 REST API, the maximum I could upload was around 5GB which is what even Amazon says their maximum limit for direct upload is. But when tried uploading a file which larger then the limit it throws an exception "Your proposed upload exceeds the maximum allowed size" and also has a HTTP response returned where the header tag 'transfer-encoding' is 'chunked'.
Here's a randomly-selected error response from S3.
< HTTP/1.1 412 Precondition Failed
< x-amz-request-id: 207CAFB3CEXAMPLE
< x-amz-id-2: EXAMPLE/DCHbRTTnpavsMQIg/KRRnoEXAMPLEBJQrqR1TuaRy0SHEXAMPLE5otPHRZw4EXAMPLE=
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Fri, 23 Jun 2017 19:51:52 GMT
< Server: AmazonS3
<
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>...
The Transfer-Encoding: chunked response header only indicates that the error response body S3 is sending back to you will use chunked transfer encoding.
This is unrelated to what is permitted for uploads, and the presence of Transfer-Encoding: chunked in either direction (request or response) of an HTTP transaction is independent of whether it is present or supported in the opposite direction.
The PUT object REST API call does not support Transfer-Encoding: chunked on the request. It requires Content-Length: in the request headers, which precludes using chunked transfer encoding.
There is no chunking, blocking, etc., mechanism involved at the HTTP layer in standard uploads -- there is no meaningful internal structure "part-size," because there are no parts: it's a continuous TCP stream of un-encoded octets of exactly Content-Length length (number of octets/bytes), with retries and network errors handled by TCP, and HTTP unaware of these mechanisms.
If the Content-Length header you send exceeds the maximum allowed upload, you get the error about your proposed upload exceeding the maximum allowed size. If the connection is accidentally or intentionally severed before Content-Length number of octets are received by S3, the uploaded data is discarded, because partial objects are never created.