I have a pipeline that gives http response (using http client origin for GET request) in json format which uses pagination option ‘By page Number’. After reaching the final page, pipeline doesn’t come to a halt but instead keeps running and writes the last page records over and over again. We use batch mode, but tried with polling and streaming modes but no luck. Any specific settings to be followed?
Related
I am passing json data to a post request via a web task.
The body contains a number of records that are to be saved by the api.
The body returned contains a status per record sent, but within the webtask i cannot see the over all status of the post request.
In postman, the overall status shows up like below:
This status and response code (429 in this case) is not visible in the output of the webtask.
Is anyone aware if i can view this in the webtask, as its clearly visible via postman.
Thanks,
I don't believe there's any way to get this; some time ago the MS response was to create a fn app to capture it and call that from adf :)
However you can at least distinguish 2xx from 4xx by using the On Success and On Failure dependencies in the pipeline
Is it possible to intercept the request going through Charles and immediately return 500 error code without sending this request to the server?
Can't find any information on this. All resources suggest to wait for the response and then change HTTP response code to 500.
I assume you have already tried adding a rewrite rule to make the request to be returned with the 500 status. Have you tried combining this with a map local, to an empty file on your disk, for instance? It may work.
If this doesn't work too, I think I would do a Map Remote to another path on my localhost (for instance: http://localhost:8081/exected-response-500) and make that URL to return the 500 status error (in my case I would use a basic Spring Boot app to achieve this).
We are working on a new RESTful api using ASP.NET Web API. Many of our customers receive a nightly datafeed from us. For each feed we run a schedule SQL Agent job that fires off a stored procedure which executes an SSIS package and delivers files via Email/FTP. Several customers would benefit from being able to run this job on demand and then receive either their binary file (xml, xls, csv, txt, etc.) or a direct transfer of the data in JSON or XML.
The main issue is that the feeds generally take a while to run. Most run within a few minutes but there are a couple that can take 20 minutes (part of the project is optimizing these jobs). I need some help finding a best practice for setting up this api.
Here are our actions and proposed REST calls
Create Feed Request
POST ./api/feedRequest
Status 201 Created
Returns feedID in the body (JSON or XML)
We thought POST would be the correct request type because we're creating a new request.
Poll Feed Status
GET ./api/feedRequest/{feedID}
Status 102 Processing (feed is processing)
Status 200 OK (feed is completed)
Cancel Feed Request
DELETE .api/feedRequest/{feedID}
Status 204 No Content
Cancels feed request.
Get Feed
GET .api/feed/{feedID}
Status 200 OK
This will return the feed data. We'll probably pass parameters into the header to specify how they want their data. Setting feedType to "direct" would require JSON or XML setting in Content-Type. Setting feedType to "xml", "xls", "csv", etc., will transfer a binary data file back to the user. For some feeds this is a custom template that is specified in the feed definition already stored in our tables.
Questions
Does it appear that we're on the right track? Any immediate suggestions or concerns?
We are trying to decide whether to have a /feed resource and a /feedRequest resource or whether to keep it all under /feed. The above scenario is the two resource approach. The single resource would POST /feed to start request, PUT /feed to check the status, GET /feed when it's done. The PUT doesn't feel right and right now we're leaning towards the stated solution above. Does this seem right?
We're concerned about very large dataset returns. Should we be breaking these into pieces or will REST service handle these large returns. Some feeds can be in excess of 100MB.
We also have images that may be generated to accompany the feed, they're zipped up in a separate file when the feed stored procedure and package are called. We can keep this all in the same request and call GET /feed/{feedID}/images on the return.
Does anyone know of a Best Practice or a good GitHub example we could look at that does something similar to this with MS technologies? (We considered moving to ASP.NET Core as well.
I want to test the performance of graphics API, but Jmeter shows me the error of 400(Bad request) every time.
what I am doing is this:-
1.Created an Http Header Manager and provided respective content type and tokens
2.Created an HTTP Request
In Body:- Provided Graphql query(formatted query which postman accept)
3.Added a Listener
Every time it is showing Bad request but the same process works fine with Postman.
Please help.
If your requests works fine in Postman you can just record it with JMeter
Prepare JMeter for recording. The easiest way is using JMeter Templates feature.
From JMeter's main menu choose `File -> Templates -> Recording" and click "Create"
Click HTTP(S) Test Script Recorder and press "Start"
Prepare Postman for recording. Configure it to use JMeter as a Global Proxy
Execute your test in Postman
JMeter will capture the request and store it under Thread Group -> Recording Controller
I am developing a project which can disable a particular REST URI at runtime so that consumers will get an error. I have created interface/controller for disabling a URI. But I can not figure out how to disable it.
I thought of setting disable flag for the URI and sending blank data, but consumer needs to see an error.
I can not use Response every where.
Need Help
Implement a ContainerRequestFilter(more on filters), which when a specific criteria is met will call:
ContainerRequestContext.abortWith(javax.ws.rs.core.Response)
with a proper HTTP status code.