Using this search api:
https://api.github.com/search/commits?q=%22sentiment%22&per_page=100&page=4
It is possible to receive results for commits for a specific keyword
Are there any additional query parameter which could added and not only using the per_page and page? For example here there is the since and until query parameters but I found them only here is for a /repos/{owner}/{repo}/commits option
Related
I have a API source in an ADF DataFlow task. The API source gives me the current page and the toatl number of pages in the body of the response. I want to use that information to paginate through my API source. I'm able to paginate through it just fine outside of a DataFlow activity using the range function. The issue is that the Rest transformation in a DataFlow activity does not support the range function. I've been trying to use the AbsoluteUrl function plus an expression to do add one to the current page returned by the body but either pagination does not accept expressions or I cannot figure out the syntax
I have a url like this:
BaseURL/fabricationcodes?facets=relatedArticles:Not%20Empty&page={PageNumber}.
In this example my rest linked service URL has everything I need minus the &page=pageNumber. So I'm trying to add that part with the key/value pair function of AbsoluteUrl. The Key being &page= and the value should be currentPage +1. My desire is for it to get the first page, page 0 and then add +1 to that to formulate the next pages url. the end condition being when body.totalPages == body.currentPage
I've tried a bunch of different expression formaulations but none seem to work and debugging in a Data flow is tough b/c the logging and error messaging is poor
What I have right now.
As data flow don't support Range option or you cannot use dynamic expression to get page from API response.
To work around the issue, you can use Data Flow activity within ForEach loop using range function in dynamic expression.
First take a web activity and pass the URL of the Rest API as below Ito get the total no of Pages from API response
then take a for each activity to iterate on API like pagination give the Dynamic expression as #range(1,activity('Web1').output.total_pages)
I will iterate the API till the respective range in sequential manner.
create parameter with type string in source DataSource.
give that parameter as dynamic value in relative URL.
after this gave parameter value as ?page=#{item()} to give the no coming from range to the page.
OUTPUT:
i am trying to query all work items with a link to pull request.
I cant find any External Link operation except count .
In gui there is no such option .
Can i use WIQL to create a query to get all work items with pull request ?
Thanks
As of this time, it is not supported to query external links by WIQL. WIQL can only query work links.
I have a project scenario to get all values from an endpoint URL. I am using ADF Pipeline but I'm having some issues with pagination.
To get the following values, I need to make requests with the PaginationCursor value in the current body response in the following request header.
I have read that ADF supports the following case, which would be mine.
Next request’s header = property value in current response body ADF - Pagination support
I don't know how to use the following attributes in order to use the paginationCursor value from the current response body in the header of the next request.
Attributes for pagination in ADF
I tried to reproduce above but not successful. Instead, if you want to do it without pagination, you can try this approach.
First create a web activity with any page URL of your API to get the total number of pages count.
In ForEach create an array for page numbers using the count from web activity as
#range(1,activity('Web1').output.total_pages)
Inside ForEach use the copy activity and give the source REST dataset parameter for the page number like ?page=#{item()}.
In the sink dataset also, create a dataset for each page with the dataset parameter value like APIdataset#{item()}.csv. This generates the sink dataset names like APIdataset1.csv, APIdataset2.csv,...
Now, you can copy from your REST API without pagination.
My repro for your reference:
Copy activity:
I could solve this problem with the following attributes.
Solution
In the Headers I had to put the name of the header of the next call. In my case the name is PaginationCursor and I got the value of this header from the actual body response called paginationCursor.
I wonder how can I send a GET request to GitHub search API, specifically https://api.github.com/search/repositories and make the query to include several languages instead of one.
Here's my current query.
https://api.github.com/search/repositories?q=stars:%3E=1000+language:scala&sort=stars&order=desc&per_page=10
I have tried doing something like this but it didn't work as well
https://api.github.com/search/repositories?q=stars:%3E=1000+language:[scala, java]&sort=stars&order=desc&per_page=10
Thanks for your help
You need to pass in multiple language: element for being able to pass multiple languages to the query as per the doc.
For your specific case, the query would be :
https://api.github.com/search/repositories?q=stars:%3E=1000+language:scala+language:java&sort=stars&order=desc
with pagination applied it would be :
https://api.github.com/search/repositories?q=stars:%3E=1000+language:scala+language:java&sort=stars&order=desc&per_page=10
However, with pagination applied your search results will be limited in the browser.
I would like to list public GitHub repositories with the latest create/update/push timestamps (for me any of these is acceptable). Can I achieve this with the GitHub API?
I have tried the following:
Tried using /repositories endpoint, and use the link header to navigate to the last page. However, the link header I receive only has first and next links, whereas I need a last link.
Tried using /search/repositories endpoint. This will work as long as I have a keyword or filter in the q parameter, but it will not accept an empty q parameter.
I got in touch with GitHub support, and there are two solutions to this:
Use binary search on the since parameter of the /repositories endpoint to find the last page.
Cons: may quickly exhaust the API rate limit.
Use the /search/repositories endpoint with an always-true predicate such as stars>=0.
Cons: likely to cause a query timeout/ incomplete results.