I have a list of entity IDs that I need to fetch from Dynamics into my application. Using the WebAPI the query must be URL encoded and passed as a GET request which limits the number of entities to about 130 (more than that hits the URL length limit)
Previously we could use the SOAP endpoint to make a single POST request but this is on the deprecation path.
So what is the recommended way to make such a query now?
Some options:
Manually paginate and run through the WebAPI making parallel requests. This is less than ideal, especially when making abstractions around Dynamics since requests may fail.
UPDATE
Thanks to #Guido Preite I was able to use a WebAPI batch request. Basically you issue the GET request with the 130+ entities but encode it in the batch request body to work around the character limit. I'll post the request below for Postman ...
Post to {{myDynamicsURL}}/api/data/v9.0/$batch
Headers
Authorization:Bearer {{token}}
Content-Type:multipart/mixed;boundary=batch_AAA123
Accept:application/json
Cache-Control:no-cache
OData-MaxVersion:4.0
OData-Version:4.0
Prefer:odata.maxpagesize=500, odata.include-annotations="*", return=representation
Body
--batch_AAA123
Content-Type: application/http
Content-Transfer-Encoding:binary
GET {{myDynamicsURL}}/connections?$filter=(connectionid eq 9b704176-2f60-e911-a830-000d3a385a17 or connectionid eq 1ccc526c-2e6c-e911-a831-000d3a385a17 ... <REST OF QUERIES HERE>) HTTP/1.1
Accept: application/json
--batch_AAA123--
you can query using a FetchXML passed inside a POST request
https://dreamingincrm.com/2017/01/15/executing-large-fetchxml-with-webapi/
Related
I am working on a POST HTTP API which does not modify or create any state on the server. The API is implemented with method POST as it needs to accept multiple complex inputs which would not be possible using query parameters.
What is the correct response status to return in case of conditional check failures (If-Match/If-None-Match) for such read-only POST APIs, should it be 304 Not Modified or 412 Precondition Failed?
Note: This is an internal service API where the client is aware that it is a non modifying request.
For a GET request, we would expect an If-None-Match header, which would normally produce a 200 response with an updated copy of the representation if the condition holds, and a 304 response when the precondition fails (meaning that the clients copy of the resource is already up to date).
The semantics should be the same when we use POST in a read-only way. (Note that we are in a bit of an edge case here; a general purpose client won't normally know that this particular POST request is safe, and probably won't know which precondition headers to attach to the request. For instance, if you try to use an HTML form in a web browser to access this resource, you probably aren't going to get the conditional headers that you want.)
412 Precondition Failed is normally used when requesting a modification to the resource, in combination with an If-Match header.
Reference: HTTP Semantics, section 13.
I'm working with Django REST framework. By default, all the requests returns a JSON object containing the pagination (prev, next, count, results). This is useful in 90% of the cases where the user retrieves or creates info about something. However, there are a few resources which don't have to return anything but rather a confirmation that everything went smoothly - for example, imagine a resource which is just a heartbeat request ("ping") to maintain the session active.
Would it be okay to return a simple response, such as {result: true} (without any pagination like the rest of resources have) or would this be an eventual violation of the REST principles?
If all you want is to know if the URI is fully serviceable, disregarding the body completely, you should simply support a HEAD request instead of GET.
Yes, ouf course such a response to a ping request is OK. Pagination is something only suitable for a collection that can be paged. Paging An unique resource that is not a collection does not make sense.
For the ping request you could even leave the response body empty.
Request:
GET /ping
Response:
200 OK
Content-Length: 0
Pagination should be solved with range headers or hyperlinks. You don't need the body in empty responses, just the status header.
Many REST APIs provide the ability to search for resources.
For example, resources of type A may be fetched using the following HTTP request:
GET /A?prop1={value1}&prop2={value2}
I'm using optimistic locking and therefore would like to return a version for every returned resource of type A. Until now, I used the ETag header when fetching only one resource using its ID.
Is there an HTTP way for returning version for multiple resources in the same response? If not, should I include the versions in the body?
Thanks,
Mickael
EDIT: I found on the web that the ETag is often generated by computing a hash of part of the reply. This approach fits well with my case since a hash of the returned collection will be computed. However, if the client decides to update one of the elements in the collection, which ETag should he put in the If-Match header? I'm thinking that including the ETags of the individual elements is the only solution...
I would adopt one of these options:
Make ETags weak by default and they are generated with the resource current state, not with the resource representation in the HTTP response payload. With that, I can return a valid ETag for each resource in the collection query response body, besides the ETag for the whole collection in the response header.
Forget If-Match and ETags for this case and use If-Unmodified-Since with a Last-Modified supplied as a property of each resource. By doing that I can preserve the strong ETags, but clients can still make updates to one item based on the collection response without the need for another request to the resource itself.
Allow updates via PATCH on the collection resource itself, using the If-Match header with the ETag for the whole collection. This probably won't work very well if there's a lot of concurrent changes, but it's a reasonable approach.
I think it depends a little bit on the amount of resources, data and requests to reduce bandwith. But a solution could be to separate the resources in sub-requests.
Assume that the group call of GET /images?car=mustang&viewangle=front returns 5 images. Now you could include all images as binary data and the GET-request itself has a unique ETag:
GET /images?car=mustang&viewangle=front
...
HTTP 1.1 200 OK
ETag "aaaaaa"
data:image/png;base64,a123456....
data:image/png;base64,b123456....
data:image/png;base64,c123456....
data:image/png;base64,d123456....
data:image/png;base64,e123456....
The problem is now, that one added image changes the ETag of the group call and you need to transfer the complete set again altough only one image has changed:
GET /images?car=mustang&viewangle=front
If-None-Match "aaaaaa"
...
HTTP 1.1 200 OK
ETag "bbbbbb"
data:image/png;base64,a123456....
data:image/png;base64,b123456....
data:image/png;base64,c123456....
data:image/png;base64,d123456....
data:image/png;base64,e123456....
data:image/png;base64,f123456....
In this case the best solution would be that you separate the resources data from the group call. So the response includes only information for sub-requests:
GET /images?car=mustang&viewangle=front
...
HTTP 1.1 200 OK
ETag "aaaaaa"
a.jpg
b.jpg
c.jpg
d.jpg
e.jpg
By that every sub-request can be cached separatly:
GET /image/?src=a.jpg
If-None-Match "Akj5odjr"
...
HTTP 1.1 304 Not Modified
Statistics
- First request = 6x 200 OK
- Future requests if group unchanged = 1x 304 Not Modified
- Future requests if one new resource has been added = 2x 200 OK, 5x 304 Not Modified
Now I would tune the API documentation. This means the requester must check if a cache of a sub-request is available before making a call to it. This could be done by providing the ETags (or other hash) in the group request:
GET /images?car=mustang&viewangle=front
...
HTTP 1.1 200 OK
...
ETag "aaaaaa"
a.jpg;AfewrKJD
b.jpg;Bgnweidk
c.jpg;Ckirewof
d.jpg;Dt34gsd0
e.jpg;Egk29dds
f.jpg;F498wdn4
Now the requester checks the cache and finds out that a.jpg has a new ETag called Akj5odjr and f.jpg;F498wdn4 is a new entry. By that future requests are reduced:
Statistics
- First request = 6x 200 OK
- Future requests if group unchanged = 1x 304 Not Modified
- Future requests if one new resource has been added = 2x 200 OK
Conclusion
Finally you need to think about if your resources are big enough to put them in sub-requests and how often one requester repeats bis group request (so the cache is used). If not, you should include them in the group call and you do not have room for optimization.
P.S. you need to monitor all requesters to be sure all of them use caches. A possible solution would be to ban requesters calling an API URL two or more times without sending an ETag.
say we have a 'user' resource with unique constraint on 'name'. how would you design a REST API to handle a find-or-create (by name) use case? I see the following options:
option 1: multiple requests
client:
POST /user
{"name":"bob"}
server:
HTTP 409 //or something else
client:
GET /user?name=bob
server:
HTTP 200 //returns existing user
option 2: one request, two response codes
client:
POST /user
{"name":"bob"}
server:
HTTP 200 //returns existing user
(in case user is actually created, return HTTP 201 instead)
option 3: request errs but response data contains conflicting entity
client:
POST /user
{"name":"bob"}
server:
HTTP 409 //as in option1, since no CREATE took place
{"id": 1, "name":"bob"} //existing user returned
I believe the "correct" RESTful way to do this would be :
GET /user?name=bob
200: entity contains user
404: entity does not exist, so
POST /user { "name" : "bob" }
303: GET /user?name=bob
200: entity contains user
I'm also a big fan of the Post-Redirect-Get pattern, which would entail the server sending a redirect to the client with the uri of the newly created user. Your response in the POST case would then have the entity in its body with a status code of 200.
This does mean either 1 or 3 round-trips to the server. The big advantage of PRG is protecting the client from rePOSTing when a page reload occurs, but you should read more about it to decide if it's right for you.
If this is too much back-and-forth with the server, you can do option 2. This is not strictly RESTful by my reading of https://www.rfc-editor.org/rfc/rfc2616#section-9.5:
The action performed by the POST method might not result in a resource
that can be identified by a URI. In this case, either 200 (OK) or 204
(No Content) is the appropriate response status, depending on whether
or not the response includes an entity that describes the result.
If you're okay with veering away from the standard, and you're concerned about round-trips, then Option 2 is reasonable.
I am using a version of option 2. I return 201 when the resource is created, and 303 ("see other") when it is merely retrieved. I chose to do this, in part, because get_or_create doesn't seem to be a common REST idiom, and 303 is a slightly unusual response code.
I would go with Option 2 for two reasons:
First, HTTP response code, 2xx (e.g. 200 nd 201) refers to a successful operation unlike 4xx. So in both cases when find or create occurs you have a successful operation.
Second, Option 1 doubles the number of requests to the server which can be a performance hit for heavy load.
I believe a GET request which either:
returns an existing record; or
creates a new record and returns it
is the most efficient approach as discussed in my answer here: Creating user record / profile for first time sign in
It is irrelevant that the server needs to create the record before returning it in the response as explained by #Cormac Mulhall and #Blake Mitchell in REST Lazy Reference Create GET or POST?
This is also explained in Section 9.1.1 of the HTTP specification:
Naturally, it is not possible to ensure that the server does not generate side-effects as a result of performing a GET request; in fact, some dynamic resources consider that a feature. The important distinction here is that the user did not request the side-effects, so therefore cannot be held accountable for them.
As I understand, the RESTful convention is for POSTs creating a resource to return the full or annotated created entity, however it's been my experience that this entity is usually discarded unless the service itself or the client is being tested.
I'm not a slave to REST when creating public facing APIs especially when I deem that for usability or architectural reasons it doesn't make sense, but one thing I've always wondered about and never done is returning 204 No Content from POSTs creating new entities (especially ones that are large in size). This can cut down on bandwidth for users making a lot of requests and make responses on my end faster.
Is this an acceptable practice or does it make you die a little inside? Note that I wouldn't consider this without providing an endpoint to retrieve this entity for testing reasons.
EDIT: I'm looking for anecdotal observations or concrete examples of why this particular use case might be harmful, even if it was well documented.
The document you linked to has the answer to the question you are askign:
If a resource has been created on the origin server, the response
SHOULD be 201 (Created) and contain an entity which describes the
status of the request and refers to the new resource, and a Location
header (see section 14.30).
Responses to this method are not cacheable, unless the response
includes appropriate Cache-Control or Expires header fields. However,
the 303 (See Other) response can be used to direct the user agent to
retrieve a cacheable resource.
Regarding the use of 204 No Content, according to the spec, you would return this (or 200) when the POST creates a resource that is not identified by a URI. If this correctly describes your use case, then a 204 would be appropriate.
As #Dmitry refers to in his comment, the returned entity does not necessarily have to be the new resource. For example, if the resource's ID is assigned by the server, the response could be an entity containing only that server generated ID.
A concrete example of this is shown in the CouchDB documentation for POST.
For this sample request:
POST /somedatabase/ HTTP/1.0
Content-Length: 245
Content-Type: application/json
{
"Subject":"I like Plankton",
"Author":"Rusty",
"PostedDate":"2006-08-15T17:30:12-04:00",
"Tags":["plankton", "baseball", "decisions"],
"Body":"I decided today that I don't like baseball. I like plankton."
}
the server response would be an entity containing the status, ID, and revision:
HTTP/1.1 201 Created
Date: Thu, 17 Aug 2006 05:39:28 +0000GMT
Content-Type: application/json
Connection: close
{"ok":true, "id":"123BAC", "rev":"946B7D1C"}