I use a GET request with the following url to retrieve data from a sharepoint list:
https://graph.microsoft.com/v1.0/sites/{site_id}/lists/{list_id}/items?expand=fields
However, in the value-key of the response dict i get only 200 entries. The List has roughly 250 entries.
So I guess microsoft graph api has some limitation on how many samples can be requested from the endpoint at once.
I cannot find anything in the doc (https://learn.microsoft.com/en-us/graph/api/listitem-list?view=graph-rest-1.0&tabs=http) on how I could get the missing entries.
Any clues?
If the request does not contain all elements the json response contains a key #odata.nextLink.
a get request on that link gives you the next 200 entries in the list
A REST question. Let's say I have a database will a million items in it. I want to retrieve say 10,000 of them via an REST GET, passing in the GET request the ID's of the 10,000 items. Using URL request query parameters, it'll quickly exceed the maximum length of a URL. How does people solve this? Use a POST instead and pass it in the body? That seems hacky.
You should not address this form through the URL parameters, it has a limit: 2000 characters
Url limit
I guess what you are doing is something like this:
https://localhost/api/applicationcollections/(47495dde-67d2-4854-add0-7e25b2fe88e4,1b470799-cc8a-4284-9ca7-76dc34a5aebb)
If you are planning to get more than 10k records you can pass the information on the body of the request which doesn't have any limit. Technically speaking you should do it through a POST request, but that is not the intent with the semantic of the POST verb. Even for the GET you can include a body:HTTP GET with request body but it should not consider as part of the semantic.
Normally you don't filter 10k elements by id, instead, you get 10k elements on a request, passing a pagination parameter if you want through the URL, but that can kill your app, especially considering that the DTO has more than one field, like
ApplicationDto
field1
field2....
field15
Bellow, you have an example of how to pass pagination parameters and get the first 10k records
https://localhost:44390/api/applications?pageNumber=1&pageSize=10000
Also, the APIs should return an extra header, let's call it X-Pagination where you can get the information if you have more pages to paginate, including as well the total amount of elements.
As an extra effort to reduce the size of the request, you can shape the data and only get the fields you need.
ApplicationDto should bring only: field1, field3 see bellow:
https://localhost:44390/api/applications?fields=field1,field3
See how Twitter address this problem as well:
Twitter cursor response
Hope this helps
I can access both the RSA REST and WSDL based APIs. Their documents refer to content and resourses. However, when logging into the actual tool I can access things like incident ticket and the like.
Incidents tickets will have INC-1234 which would be a string. But the API takes integer values for either content or resource when searching by ID.
self.wsdl['search'].service.SearchRecordsByReport(
sessionToken=self.token,
reportIdOrGuid=14,
pageNumber=1
)
If I don't provide an integer i get a 404 with a simple
"Message": "The request is invalid." in the REST
and something a little more complex if i pass a value like that in SOAP
So, Question 1. is there an RSA resource that correlates/demystifies the relationships of calls in the API with the actual names of things I'm dealing with the in the UI.
Question 2. Is there a code library were someone has created/modified rsa incident tickets before out there?
If you're talking about getting the record contentID you can get it two ways, in the GUI hover over your key field (the clickable link) and in the bottom left corner of archer it'll show something similar to "123456|100|17" where the first set is the contentID of that record.
If you're talking about getting a reportID (14 like you've shown above) you can do the same thing when you go to the master reports list, and it'll show in the bottom right corner of your browser.
If you want to do it all from code I usually piece it together like this:
Execute a search of a report; Get the field IDs from the report <FieldDefinition> (it's probably important to make sure your keyfield is included on the report GUI so you'll be able to identify which record you're dealing with); make another API call (probably REST since it really handles the metadata) to get the field name etc. (or you could make it all static and just hard-code the field names with the corresponding GUIDS)
Within the <Record> tag, you can snag the contentID attribute. Then take a step down and compare the field tag that matches your keyfield and you'll have the contentID of that record.
Make record updates, deletes, etc whatever you need to do.
If you're looking for a specific report ID, make a REST call to get all reports, and it'll return report names and IDs, and then go about your business making your other calls.
I've recently came across similar situation where I have to search for a field value (e.g. "INC-1234") using Archer's API.
So, for your Question 1:
The method that I found convenient to my requirements is "ExecuteQuickSearchWithModuleIds" which can search for a value and provides all ContentIDs that have those values (given the ModuleId).
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<ExecuteQuickSearchWithModuleIds xmlns="http://archer-tech.com/webservices/">
<sessionToken>ABC123ABC123ABC123ABC123ABC1</sessionToken>
<moduleIds>123</moduleIds>
<keywords>INC-1234</keywords>
<pageNumber>9999999</pageNumber>
<pageSize>9999999</pageSize>
</ExecuteQuickSearchWithModuleIds>
</soap:Body>
</soap:Envelope>
And for your Question 2, I found a rich library for Python3 Archer RSA.
Hope this helps someone who lands on this question.
We are trying to post a really long query string, that exceeds 2048 characters to sharepoint rest api.I am making a post to the below api
http://sitename/_api/web/sitegroups?$select=LoginName,Id,Title&$Filter=LoginName%20eq%20'Test_Grp'
We modified the config file of our sharepoint site to the below,
<system.web>
<httpRuntime maxQueryStringLength="65536"/>
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxQueryString="65536" />
But still sharepoint throws a 400 error stating that the maxquery length is exceeded. Do I have to change the configurations in some other place?
Is it possible to send the query string as part of the http post body so that the query string length could be let alone ?
SharePoint REST supports maximum query length of 256 characters
excluding query options.
Looking at your REST query, there is not issue. Please send the error message you are getting to resolve your query.
According to REST style, it's generally assumed that HTTP POST, GET, PUT, and DELETE methods should be used for CREATE, READ, UPDATE and DELETE (CRUD) operations.
But if we stick to the HTTP method definitions, it might not be so clear.
In this article it's explained that:
In a nutshell: use PUT if and only if you know both the URL where the resource will live, and the entirety of the contents of the resource. Otherwise, use POST.
Mainly because
PUT is a much more restrictive verb. It takes a complete resource and stores it at the given URL. If there was a resource there previously, it is replaced; if not, a new one is created. These properties support idempotence, which a naive create or update operation might not. I suspect this may be why PUT is defined the way it is; it's an idempotent operation which allows the client to send information to the server.
In my case I usually issue updates passing all the resource data, so I could use PUT for updates, but every time I issue an update I save a LastUser and LastUpdate column, with the user id that made the modification and the time of the operation.
I'd like to know your opinion, because strictly speaking those two columns are not part of the resource, but they do prevent the operation from being idempotent.
Ignoring the comment about the REST style mapping CRUD to the HTTP methods, this is an excellent question.
The answer to your question is, yes you are free to use PUT in this scenario even though there are some elements of the resource that are updated by the server in a non-idempotent manner. Unfortunately, the reasoning behind the answer is quite vague. The important thing, is to understand what was the intent of the client request. The client intended to completely replace the contents of resource with the values passed. The client is not responsible for the server doing other operations and therefore the semantics of the HTTP method are not violated.
This is the reasoning that is used to allow a server to update a page counter when you do a GET operation. The client didn't ask for an update therefore the GET is safe even though the server chose to make an update.
The whole, complete resource versus partial resource debate has finally been spelled out in an update to the HTTP spec
An origin server SHOULD reject any PUT
request that contains a
Content-Range header field, since it
might be misinterpreted as partial
content (or might be partial content
that is being mistakenly PUT as a
full representation). Partial content
updates are possible by targeting a
separately identified resource with
state that overlaps a portion of
the larger resource, or by using a
different method that has been
specifically defined for partial
updates (for example, the PATCH
method defined in [RFC5789]).
So, what we are supposed to do is now clear. What is not so clear is why there exists this constraint on only being allowed to send full responses. That question has been asked and IMHO remains unanswered in this thread on rest-discuss.
As LastUser and LastUpdate are not modifiable by the client, I'd remove them from the representation of your resource altogether. Let me explain my reasoning with an example.
Let's say that our typical example API will return the following representation to the client when asked to provide a single resource:
GET /example/123
<?xml version="1.0" encoding="UTF-8" ?>
<example>
<id>123</id>
<lorem>ipsum</lorem>
<dolor>sit amet</dolor>
<lastUser uri="/user/321">321</lastUser>
<lastUpdate>2011-04-16 20:00:00 GMT</lastUpdate>
</example>
If a client wants to modify the resource, it would presumably take the whole representation and send it back to the API.
PUT /example/123
<?xml version="1.0" encoding="UTF-8" ?>
<example>
<id>123</id>
<lorem>foobar</lorem>
<dolor>foobaz</dolor>
<lastUser>322</lastUser>
<lastUpdate>2011-04-16 20:46:15 GMT+2</lastUpdate>
</example>
Since the API generates values for lastUser and lastUpdate automatically and cannot accept data provided by the client, the most appropriate response would be 400 Bad Request or 403 Forbidden (since the client cannot modify these values).
If we want to be compliant with REST and send a full representation of the resource when doing a PUT request, we need to remove lastUser and lastUpdate from the representation of the resource. This will allow clients to send the full entity via PUT:
PUT /example/123
<?xml version="1.0" encoding="UTF-8" ?>
<example>
<id>123</id>
<lorem>foobar</lorem>
<dolor>foobaz</dolor>
</example>
The server would accept a full representation now that it doesn't contain lastUpdate and lastUser.
The question that remains is how to provide clients with access to lastUpdate and lastUser. If they don't need it (and these fields are required just internally by the API), we are fine and our solution is perfectly RESTful. If however clients need access to this data, the cleanest approach would be to use HTTP headers:
GET /example/123
...
Last-Modified: Sat, 16 Apr 2011 18:46:15 GMT
X-Last-User: /user/322
...
<?xml version="1.0" encoding="UTF-8" ?>
<example>
<id>123</id>
<lorem>foobar</lorem>
<dolor>foobaz</dolor>
</example>
Using a custom HTTP header is not ideal because user agents need to be taught on how to read it. If we want to provide clients with access to the same data in a more easier way, the only thing that we can do is to put the data into the representation, and we are facing the same problem as in your original question. I would at least try to mitigate it somehow. If the content type used by the API is XML, we can put the data into node attributes instead of exposing them directly as node values, i.e.:
GET /example/123
...
Last-Modified: Sat, 16 Apr 2011 18:46:15 GMT
...
<?xml version="1.0" encoding="UTF-8" ?>
<example last-update="2011-04-16 18:46:15 GMT" last-user="/user/322">
<id>123</id>
<lorem>foobar</lorem>
<dolor>foobaz</dolor>
</example>
This way we'll at least avoid the problem where a client would attempt to submit all XML nodes in a follow-up PUT request. This won't work with JSON, and the solution is still a bit on the edge of idempotency (since the API would still have to ignore the XML attributes when processing the request).
Even better, as Jonah pointed out in the comments, if clients need access to lastUser and lastUpdate, these can be exposed as a new resource, linked from the original one e.g. as follows:
GET /example/123
<?xml version="1.0" encoding="UTF-8" ?>
<example>
<id>123</id>
<lorem>foobar</lorem>
<dolor>foobaz</dolor>
<lastUpdateUri>/example/123/last-update</lastUpdateUri>
</example>
... and then:
GET /example/123/last-update
<?xml version="1.0" encoding="UTF-8" ?>
<lastUpdate>
<resourceUri>/example/123</resourceUri>
<updatedBy uri="/user/321">321</updatedBy>
<updatedAt>2011-04-16 20:00:00 GMT</updatedAt>
</lastUpdate>
(The above can be also nicely expanded to provide a full audit log with individual changes, providing a resource changelog is available.)
Please note:
I agree with Darrel Miller's take on the question, but I wanted to provide a different approach on top of it. Note that this approach is not backed-up by any standards/RFCs/etc, it's just a different take on the problem.
The disadvantage of using PUT to create resources is that the client has to provide the unique ID that represents the object it is creating. While it usually possible for the client to generate this unique ID, most application designers prefer that their servers (usually through their databases) create this ID. In most cases we want our server to control the generation of resource IDs. So what do we do? We can switch to using POST instead of PUT.
So:
Put = UPDATE
Post = INSERT
Hopefully, this helps for your specific case.
The HTTP methods POST and PUT aren't the HTTP equivalent of the CRUD's create and update. They both serve a different purpose. It's quite possible, valid and even preferred in some occasions, to use PUT to create resources, or use POST to update resources.
Use PUT when you can update a resource completely through a specific resource. For instance, if you know that an article resides at http://example.org/article/1234, you can PUT a new resource representation of this article directly through a PUT on this URL.
If you do not know the actual resource location, for instance, when you add a new article, but do not have any idea where to store it, you can POST it to an URL, and let the server decide the actual URL.