Normally when I want to page results from a REST API endpoint, I would use $top + $skip.
However, when the endpoint wraps a Generic Inquiry, $top + $skip no longer affect the returned results. ($filter still works)
My goal is to export data from [GLTran], so given the high number of records in this table, I need to be able to page the results.
Is there a way to do this? Or is there a better way to export all columns from [GLTran]?
There is no option available to page the GI results via the API. Your best bet is to configure Parameters for a GI the way it shows GLTran for several GL batches (say from Batch X to Batch Y of Module Z). Then you simply PUT parameter values through the API call and export GLTran records in batches.
I also highly recommend you to check the Exporting Records from Acumatica via REST Contract-Based API topic for examples on how to implement pagination on multiple REST requests. The $skip query option does not skip the records exported by the API from Acumatica, what in fact it does is excluding the first N records from the resultset returned by the API.
Related
I have a API source in an ADF DataFlow task. The API source gives me the current page and the toatl number of pages in the body of the response. I want to use that information to paginate through my API source. I'm able to paginate through it just fine outside of a DataFlow activity using the range function. The issue is that the Rest transformation in a DataFlow activity does not support the range function. I've been trying to use the AbsoluteUrl function plus an expression to do add one to the current page returned by the body but either pagination does not accept expressions or I cannot figure out the syntax
I have a url like this:
BaseURL/fabricationcodes?facets=relatedArticles:Not%20Empty&page={PageNumber}.
In this example my rest linked service URL has everything I need minus the &page=pageNumber. So I'm trying to add that part with the key/value pair function of AbsoluteUrl. The Key being &page= and the value should be currentPage +1. My desire is for it to get the first page, page 0 and then add +1 to that to formulate the next pages url. the end condition being when body.totalPages == body.currentPage
I've tried a bunch of different expression formaulations but none seem to work and debugging in a Data flow is tough b/c the logging and error messaging is poor
What I have right now.
As data flow don't support Range option or you cannot use dynamic expression to get page from API response.
To work around the issue, you can use Data Flow activity within ForEach loop using range function in dynamic expression.
First take a web activity and pass the URL of the Rest API as below Ito get the total no of Pages from API response
then take a for each activity to iterate on API like pagination give the Dynamic expression as #range(1,activity('Web1').output.total_pages)
I will iterate the API till the respective range in sequential manner.
create parameter with type string in source DataSource.
give that parameter as dynamic value in relative URL.
after this gave parameter value as ?page=#{item()} to give the no coming from range to the page.
OUTPUT:
I want download WIQL report using REST API. REST response doesn't give all fields but it gives a list of URLS as workItems.
To get field values I need to download each WorkItem separately.
Any direct REST way to accomplish this in a single REST call?
Repeated REST calls gives me rate limiting or similar error. I get error 500 types after repeated GET request.
Genesis of this need is - There are no aggerate functions available likes of SUM, MAX, MIN, AVG Etc.
REST response doesn't give all fields but it gives a list of URLS as
workItems.
To get field values I need to download each WorkItem separately. Any
direct REST way to accomplish this in a single REST call?
Sorry but as I know, there's no direct rest api available to get WIQL report. An alternative workaround should be:
1.Use Query By Wiql to return the list of WorkItem IDs and Urls (I think this is the same rest api you use).
2.Then use Get Work Items Batch to get all the details(fields) about the requested work item ids. And here's one issue which has similar needs like yours, you can use the upgraded script from konpro11 to get list of IDs and use the IDs to get your report (with the help of second rest api).
Hope it helps :)
I have gone through following question
SharePoint REST API call with more than 500 rows
We want to use Search API with REST API or JSOM. We have large lists in SharePoint. The data can be 100000 or more items in a single list. There are multiple large lists. In some scenarios search API which will search across all lists and in some scenarios it will also search in a single lists.
So like REST API does it throw threshold error for more than 5000 matched items or does SharePoint Search REST API throws any threshold limit in any case?
As Mike pointed out, I'm not aware of any limitations in terms of API's for search in SharePoint. You will have to make a lot of sub requests to retrieve all the items if you need to do so (but generally speaking, it'd probably be better to have a more complex query, retrieving less items and giving more work to the search engine). More details on how you can paginate (REST) here.
Keep in mind that your search service application needs to be properly designed to accept all the items otherwise it might stop indexing new items. (if you're on prem. or doing hybrid)
For the Marketing API, I know that I'm able to make one call to retrieve all of the adsets from a certain account along with their insights, but am I able to specify the date_preset for the insights edge in that same call?
For example, the following gives me lifetime insights stats:
/v2.4/{accountID}/adcampaigns?fields=insights
To be clear - I know this is possible to retrieve by making separate calls for each adset id (where I know I can specify the date_preset); instead, I'd like to do this via the call where I get a long list of the ad sets plus their insights details in one go.
Yes this is possible using query expansion, however you probably should not do it in this anyway.
Using query expansion results in multiple requests being executed in one HTTP call, in this case one to get all the adcampaigns, and then N requests where N is the number of adcampaigns returned. This will in turn affect your rate limiting.
The most efficient way to request all insights for all adcampaigns (ad sets) is instead to request them at the account level, specifying aggregation level:
/v2.4/act_{ADACCOUNT_ID}/insights?date_preset=last_7_days&level=campaign
This requires just 1 request, or the number of requests to retrieve the total number of pages.
If you really want to achieve this with query expansion, you can do the following for example:
/v2.4/act_{ADACCOUNT_ID}/adcampaigns?fields=insights.date_preset(last_30_days).time_increment(all_days)
You can see the parameters to insights that would normally be query parameters of the form param_name=param_value are now in the form of param_name(param_value).
To specify the date_preset , here is the correct format . Its important to use insights as edge to get the date_preset filtering .
/v2.10/act_{ADACCOUNT_ID}/insights?fields=impressions,clicks,ctr,unique_clicks,unique_ctr,spend,cpc&date_preset=last_3d
The above one is tested with latest Graph Api version(2.10) as of now . FOr more info related to the date_preset values refer to there api docs .
https://developers.facebook.com/docs/marketing-api/insights/parameters
I have a spring batch job, which needs to fetch details from rest api call and process the particular data on my side. My rest api call will have mainly the below parameters :
StartinIdNumber(offset)
PageSize(limit)
ps: StartinIdNumber serves the same purpose as rownumber or "offset" in this particular API. The API response results are sorted by IdNumber, so by specifying a StartinIdNumber, the API will in turn perform a "where IdNumber >= StartinIdNumber order by IdNumber limit pageSize" in their DB query.
It will return the given number of user details, I need to iterate through all the ids by changing the StartingIdNumber parameter for each request.
I have seen current ItemReader implementations of spring batch framework,which read through database or xml etc. But I didn't come across any reader which helps in my case. Please suggest a way to iterate through the user details as specified above .
Note : If I write my own custom item reader, I have to take care of preserving state (last processed "StartingIdNumer") which is proving challenging to me.
Does implementing ItemStream serves my purpose? Or is there any better way?
Implementing the ItemStream interface and writing my own custom reader served my purpose. It is now state-full as required for me. Thanks.