When trying to scan HBase through the REST API in our cluster (CDH 5.14.0 | HBase 1.2.0-cdh5.14.0) the scanner's batch attribute is not working, always returns records one by one. The PUT request is as follows:
http://hbase-rest-server:20550/our-table/scanner/
With a scanner definition like:
<Scanner batch="1000" startRow="YWFh" stopRow="YmJi=" />
With the scanner id that comes in the response's location returns one record in every call in order:
http://hbase-rest-server:20550/our-table/scanner/1532043234571885278a
Why batch="1000" is not taken into account? What am I missing?
In application/octet-stream option, the scaner will ignore batching parameter.
Set Accept header to text/xml or any other options.
https://hbase.apache.org/1.2/apidocs/org/apache/hadoop/hbase/rest/package-summary.html
Related
I have a project scenario to get all values from an endpoint URL. I am using ADF Pipeline but I'm having some issues with pagination.
To get the following values, I need to make requests with the PaginationCursor value in the current body response in the following request header.
I have read that ADF supports the following case, which would be mine.
Next request’s header = property value in current response body ADF - Pagination support
I don't know how to use the following attributes in order to use the paginationCursor value from the current response body in the header of the next request.
Attributes for pagination in ADF
I tried to reproduce above but not successful. Instead, if you want to do it without pagination, you can try this approach.
First create a web activity with any page URL of your API to get the total number of pages count.
In ForEach create an array for page numbers using the count from web activity as
#range(1,activity('Web1').output.total_pages)
Inside ForEach use the copy activity and give the source REST dataset parameter for the page number like ?page=#{item()}.
In the sink dataset also, create a dataset for each page with the dataset parameter value like APIdataset#{item()}.csv. This generates the sink dataset names like APIdataset1.csv, APIdataset2.csv,...
Now, you can copy from your REST API without pagination.
My repro for your reference:
Copy activity:
I could solve this problem with the following attributes.
Solution
In the Headers I had to put the name of the header of the next call. In my case the name is PaginationCursor and I got the value of this header from the actual body response called paginationCursor.
In PowerApps, custom connector, I need to define the entire structure of the call when I create the custom connector. I know I can mark certain fields (query params) as parameters that can be filled during run time.
What I try to do is to set the value of one of my security http headers at run time.
My connector, with an API key, makes a call to the /extra/auth end point
/extra/auth returns another key value.
I need to use this new key value in any consecutive calls to my APIs in the HEADER of the request.
I can use two separate connectors for the auth and for the application logic (which seems logic to me).
How would you go to set an http header value at run time for a custom connector - REST api?
Spring Cloud Gateway appears to be reordering my query parameters to put duplicate parameters together.
I'm trying to route some requests to one of our end points to a third party system. These requests include some query parameters that need to be in a specific order (including some duplicate parameters), or the third party system returns a 500 error, but upon receiving the initial request with the parameters in the proper order, the Spring Cloud Gateway reorders these parameters to put the duplicates together by the first instance of the parameter.
Example:
http://some-url.com/a/path/here?foo=bar&anotherParam=paramValue2&aThirdParam=paramValue3&foo=bar
Becomes:
http://some-url.com/a/path/here?foo=bar&foo=bar&anotherParam=paramValue2&aThirdParam=paramValue3
Where the last parameter was moved to be by the first parameter because they had the same name.
The actual request output I need is for the query parameters to be passed through without change.
The issue lays in the UriComponentsBuilder which is used in RouteToRequestFilter.
UriComponentsBuilder.fromUri(uri) is going to build up a map of query params. Because this is a LinkedMultiValueMap you see the reordering of the used query params.
Note that RFC3986 contains the following
The query component contains non-hierarchical data that, along with data in the path component (Section 3.3), serves to identify a resource within the scope of the URI’s scheme and naming authority (if any).
Therefor I don’t think there needs to be a fix in Spring Cloud Gateway.
In order to fix this in your gateway, you'll need to add a custom filter which kicks in after the RouteToRequestFilter by setting the order to RouteToRequestUrlFilter.ROUTE_TO_URL_FILTER_ORDER + 1.
Take a look at the RouteToRequestUrlFilter how the exchange is adapted to go to the downstream URI.
Hope that helps! :)
In J.meter I need to test multiple web-services in single scenario like, after successful execution of service one it will gives session-Id and this session id will take by other services and check it and complete the scenario as per business logic?
It is classic question regarding "correlation" in JMeter. Correlation stands for the process of extracting dynamic data from previous response and passing it to the next request.
Add a JSON Extractor as a child of the first request and define a JSON Path query to extract the session-Id value and and store it into a Jmeter Variable.
Use the aforementioned JMeter Variable as the session-Id for the subsequent requests.
See API Testing With JMeter and the JSON Extractor article for comprehensive instructions.
One of the external .NET desktop application is supposed to update our internal MySQL database (Couple of inserts and updates). We’ve agreed to expose a REST web service in order to facilitate this.
I’m thinking of accepting the inputs as a String in JSON format and process inside my REST service. Since there’s lot of values are supposed to be passed into this web service, I’m expecting the inputs in a JSON format without having them in query string or as in params. Is this the best/acceptable way of handling these kind of cases or any other any way of designing this?
If you want insert or update a resource through rest then following is the best practices to design it.
Insert
Request: POST /resources
Body {data in json format}
Response : 201 Created
Update
Request : PUT /resources/{id}
Body {data in json format}
Response : 200 OK
Here you can replace "resources" with the term that you want to manage through rest service. And {id} in second case is the identifier which can identify the resource you want to update in your repository.