I know that it is possible to do this when I do a SQ-XML call to the api with the maxrecordcount attribute, ie:
<ns:attributes>
<ns:entry>
<ns:key>maxrecordcount</ns:key>
<ns:value>2</ns:value>
</ns:entry>
</ns:attributes>
However, the TCC only lets you change what's in the query tag, not the attributes one.
Is there any way to accomplish the equivalent of LIMIT with the TCC?
You can use pageindex and Paging Size options in Advanced parameters of sq file. look at the example in the image
Related
I'm new to playing around with calling third party REST API's.
I have an API which requires an ID (/sites/{id}/. As I don't know the ID off the top of my head and would like to query multiple ID's, is there anyway to wildcard this ID for it to run through and check for instance ID's 1 through to 10? Or is this more of a python integration?
As mentioned above, if an API happens to have a parameter "id", whether or not you can scan for all available IDs (or any ID between 1 and 10) depends entirely on the API.
In your case, the API (for help.rapid7.com) is well documented. It appears to have an endpoint to "list sites", which should give you what you're looking for:
https://help.rapid7.com/insightvm/en-us/api/index.html#operation/getSites
Sites
GET /api/3/sites
Server URL
https://help.rapid7.com/api/3/sites
Retrieves a paged resource of accessible sites.
PARAMETERS
Query Parameters
* page integer <int32>
Default: 0
The index of the page (zero-based) to retrieve.
* size integer <int32>
Default: 10
The number of records per page to retrieve.
* sort
Multiple query params of string
The criteria to sort the records by, in the format:
property[,ASC|DESC].
The default sort order is ascending.
Multiple sort criteria can be specified using multiple sort query parameters.
You would probably want to do the following:
Call /api/3/sites (with a filter) to get a list of sites you're interested in, then
Make successive calls to /sites/{id}/ for each site in the list you want detailed information about.
Hi I'm trying to get a sorted Collection from the Domino Rest Api. My database name is "Test/JSON_Views.nsf" and my views name "List".
The endpoint I use is
**/Test/JSON_Views.nsf/api/data/collections/name/List?sortcolumn=title&sortorder=ascending&count=20
But the JSON-Response entries aren't sorting by title in ascending order.
Should I make any settings to the column properties in the designer? If I set descending there for the title-column it works. But I want to change the sorting in my external java-application.
Is my endpoint correct? I use this Domino API Docu as Reference.
Add an additional sorting to your title column:
This gives the API the possibility to sort by title in both directions. You can do this with other columns too so you are very flexible in sorting this way.
The doc says that if the column isn't sorted in design then the sortcolumn parameter has no effect, so the answer is "Yes" you should change the design of the desired column. If doing that is unworkable in whatever context you use it, then create a second view and use that instead.
i am working on aem 6.3 and would like to get page name
SELECT * FROM [cq:Page] WHERE ISDESCENDANTNODE("/content/Product/Silhouettes/Accessories/Bands/Headband")
If I need to retrieve name of the nodes using sql-2 , how do I achieve it?
You can specify column constraints like title, node name, etc this way -
SELECT nodeSet.name, nodeset.title
FROM [cq:Page] AS nodeSet
WHERE ISDESCENDANTNODE("/content/Product/Silhouettes/Accessories/Bands/Headband")
Note: the query tool in AEM(Tools -> Query) will not list the query results according to the columns you've mentioned, it will only list the node paths.
You can look at using /etc/importers/bulkeditor.html or AEM fiddle tool to visualize the query results based on column constraints.
If you want to achieve this programmatically, you can use the same query as you've mentioned in your question and use javax.jcr.query.* and javax.jcr.Node.* API's to retrieve just about any property from the query result. This article here should help you achieve this programatically.
Use the ResourceResolver API to execute and obtain the query results:
final Iterator<Resource> pagesIterator = resolver.findResources('<your_query_here>', javax.jcr.query.Query.JCR_SQL2);
while (pagesIterator.hasNext()) {
final Resource pageResource = pagesIterator.next();
LOG.info(pageResource.getName());
}
However, please note that if you are using any version higher then CQ 5.6, you should use instead the Page API.
In this case, the listChildren(Filter<Page> filter, boolean deep) method will do the job.
The PageFilter parameter may be used if you want to filter through pages of some kind. So if no extra criteria for your page finding algorithm, your may pass null or a new empty object.
The boolean parameter: if false it returns only direct child pages, and if true would list all the descendant pages of the given page.
Therefore, the equivalent solution of the SQL Query which would provide you the same end results would be:
Iterator<Page> rootPageIterator = rootPage.listChildren(null, true);
I am following a sample for Azure DocumentDB below. In the sample, C# code queries for documents in the DocumentDB.
https://github.com/Azure/azure-documentdb-dotnet/blob/master/samples/rest-from-.net/Program.cs
Line 182:
var qry = new SqlQuerySpec { query = "SELECT * FROM root" };
var r = client.PostWithNoCharSetAsync(new Uri(baseUri, resourceLink), qry).Result;
The problem is the result 'r' only contains the first 100 documents. If I use the Client SDK, I can get more than 100. I tried using stream, but had no luck so far. Any help would be appreciated!
For a SQL query the results are returned in segments if the result set is too large. The results are returned in chunks of 100 items or 1 MB (whichever limit is hit first) by default.
You can either use continuation tokens to get each segment after another. Or you set the x-ms-max-item-count custom header in a request to increase the limit to an appropriate value.
You can have a look the the REST API for further details.
For the sample program you have to add the line
client.DefaultRequestHeaders.Add("x-ms-max-item-count", "1000");
in order to get 1000 documents instead of 100.
I'm just guessing here, but it might be worth a shot. Here's the documentation from MSDN that describes the List action:
https://learn.microsoft.com/en-us/rest/api/documentdb/list-documents
In the "Headers" section under "Response" it is mentioned that you might get an optional token in the header "x-ms-continuation". Based on the description you have to issue another GET request with this token specified to get the other elements of the result set.
Can you check whether you get a header like this in the response? If so, you can issue another get request with this token specified (see the same documentation page under "Request").
This is similar to to this question which doesn't have any answers. I've read all about how to use cursors with the twitter, facebook, and disqus api's and also this article about how disqus generally built their cursors, but I still cannot seem to grok the concept of how they work and how to implement a similar solution in my own projects. Can someone explain specifically the different techniques and concepts behind them?
Lets first understand why offset pagination fails for large data sets with an example.
Clients provide two parameters limit for number of results and offset and for page offset.
For example, with offset = 40, limit = 20, we can tell the database to return the next 20 items, skipping the first 40.
Drawbacks:
Using LIMIT OFFSET doesn’t scale well for large
datasets. As the offset increases the farther you go within the
dataset, the database still has to read up to offset + count rows
from disk, before discarding the offset and only returning count
rows.
If items are being written to the dataset at a high frequency, the
page window becomes unreliable, potentially skipping or returning
duplicate results.
How Cursors solve this ?
Cursor-based pagination works by returning a pointer to a specific item in the dataset. On subsequent requests, the server returns results after the given pointer.
We will use parameters next_cursor along with limit as the parameters provided by client in this case.
Let’s assume we want to paginate from the most recent user to the oldest user.When client request for the first time , suppose we select the first page through query:
SELECT * FROM users
WHERE team_id = %team_id
ORDER BY id DESC
LIMIT %limit
Where limit is equal to limit plus one, to fetch one more result than the count specified by the client. The extra result isn’t returned in the result set, but we use the ID of the value as the next_cursor.
The response from the server would be:
{
"users": [...],
"next_cursor": "1234", # the user id of the extra result
}
The client would then provide next_cursor as cursor in the second request.
SELECT * FROM users
WHERE team_id = %team_id
AND id <= %cursor
ORDER BY id DESC
LIMIT %limit
With this, we’ve addressed the drawbacks of offset based pagination:
Instead of the window being calculated from scratch on each request based on the total number of items, we’re always fetching the next count rows after a specific reference point. If items are being written to the dataset at a high frequency, the overall position of the cursor in the set might change, but the pagination window adjusts accordingly.
This will scale well for large datasets. We’re using a WHERE clause to fetch rows with id values less than the last id from the previous page. This lets us leverage the index on the column and the database doesn’t have to read any rows that we’ve already seen.
For detailed explanation you can visit this wonderful engineering article from slack!
Here is an article about pagination: paginating-real-time-data-cursor-based-pagination
Cursors – we need to have at least one column with unique sequential values to implement cursor based pagination. This can be similar to Twitter’s max_id parameter or Facebook’s after parameter.
In general you should pass the current item or page number in the request as a param. Other usual param is the batch size of the page. Then on the server side backend you select and return the proper dataset, with an SQL query for example.
enter image description hereHere's what I am Done with. The cursor is working as a pointer and it points to that index. and limit will pick that many rows from that pointer. Let's say we have given id 10 and limit 5 then it will go to id 10 and pick 5 elements from there.
Some Graph API connections uses cursors by default. You can use 'limit' and 'before'/'after' parameters in your call. If you are still not clear, you can post your code here and I can explain with it.