I'm currently trying to download data from an infinte streaming API endpoint (without closure) using the InvokeHTTP processor. I'm using NiFi 1.20. I'm able to connect to the API, but I'm not able to download any data. However when I'm connecting to other endpoints in the same API with a http-response that ends the connection, data streams through the processor.
Am I missing some config parameters or do I have to use another approach?
I,ve tested most of the settings in the processor. In addition I have verified that the Oauth2 token is valid by downloading from the streaming endpoint using Curl.
PS! I'm trying to dowload data from the Barentswatch API
Related
I am very new to WSO2 API manager and trying out my very first simple restful api. which returns json response and has no security since it is an internal api.
I installed WSO2 API manager locally and trying to call the rest api on my dev server which uses http and no security as I mentioned earlier.
Here is how my get url looks like:
and here is my url looks like for production and sandbox environment:
I don't have any message mediation enabled.
I went to the API store and created a trial application (so that I can get the access token. Eventhough, my dev environment api has no security, I was reading that for throttling and other purpose, I need to pass bearer token to the WSO2 api OR it will reject the request.)
When I am trying to consume the api, I get the following binary message.
Is there any way I can see the proxy log on WSO2 server so that I can see the request and its header sent to my dev server?
How can I fix this binary response to get the proper json response?
I searched all over and can't find solution to it.
You can use below steps on WSO2 ESB or APIM to enable Wire Logs.
Uncomment below line in /repository/conf/log4j.properties
log4j.logger.org.apache.synapse.transport.http.wire=DEBUG
Restart Server.
Source - http://lakshanigamage.blogspot.com/2015/03/how-to-enable-wire-logs-in-wso2-esbapim.html
I want to generate a swagger file for rest v2 connector in informatica cloud with these details.
POST CALL:
Accept: application/json
Content-Type: application/x-www-form-urlencoded
Raw Body:
token=XXXXXXX&content=record&format=csv
But informatica cloud does not have an option of application/x-www-form-urlencoded.
I am able to do the same request in POSTMAN as POSTMAN has all the functionalities.
I even tried to put the Content-Type separately in the headers section while generating the swagger file in Informatica-cloud, but still didn't work.
Someone told me to use this website: http://specgen.apistudio.io for creating the swagger file, but the site does not seem secure and thus I cannot enter any sensitive data
Is there any way I could generate the file through a website or through informatica itself?
Swagger file cannot be generated for the header “Content-Type: application/x-www-form-urlencoded” in Informatica cloud.
What can be done instead is to use 'Curl' for the rest api call in the pre/post processing command in the Mapping Task/Data Synchronization Task. You can take a look at the curl commands in here:
https://www.baeldung.com/curl-rest
Other way if you want to avoid using Curl then, you can create a 'service connector' for the REST call in the application integration.
It is also possible to run data integration tasks from application integration if you want to run them after using the service connector.
The way it works is:
Create a service connector
Create the connection for the service connector
Create a process.
Inside the process, use various services. First service can run your API connection that you just made, then you can use other service to run a data integration task which is available inside 'System service: -> Run cloud task'.
This way you can make the work done without creating a swagger file as it does not accept “Content-Type: application/x-www-form-urlencoded”.
Goal: I have an Angular 5 client that I wish to centralise logging to AWS CloudWatch so that I can view all logs (from the web client, API, lambdas, DynamoDB) in one place.
Solution considered:
(1) Create an API on the AWS Api Gateway with the CloudWatch Logs (AWS Service)
(2) Angular 5 client will post all logging to this API.
Problem:
I am having difficulty configuring the AWS Api Gateway correctly. The documentation on this is particularly sparse. I have created a simple test API with the following configuration.
Basic API configuration
This is a POST method on the API. The TestAPIRole has permission to PutEventLogs to CloudWatch.
I ran a test with the following data:
Test message
And I get the following response:
Response from test message
I think I haven't configure the API right but I am not sure what else to try. Has anyone tried to configure AWS API Gateway to post log events to Cloudwatch?
Has anyone tried to configure AWS API Gateway to post log events to Cloudwatch?
We have an error-logger API (API Gateway + Lambda) we use at work. At the minimum, it's just a simple POST endpoint that receives what the frontend (ours is in React) error handler sends. The Lambda simply writes to CloudWatch and we aggregate them through ElasticSearch.
I want to use rest api( but i don't want to use curl command , i mean i want to use invokehttp procesor for making post commands to delete queue flowfiles and start/stop processor ) I used Autorization Bearer Token like this:
eyJhbGciOiJIUzI1LiJ9.eyJzdWIiOiJzLLRraGlsYWlzaHZpbGlAQ1JBLSdLIiwiaXLzIjoiS2VyYmVyb3LQcm92aWRlciIsImL1ZCI6IStlcmJlcm9zUHJvdmlSZXIiLCJwcmVmZXJyZWRLdXLlcm5hbWUiOiJzLLRraGlsYWlzaHZpbGlAQ1JBLSdLIiwia2lSIjo1LCJleHAiOjE1MDQwMzA2LTQsImlhdCI6MTUwMzS4LzQ1LH0.VxxpvLQbGm3H475g1yScvg5B89WSws3lZUwBrUM4OvE
I have added autorization attribute in invokehttp and added link like this https://server ip/nifi-api/acess/token after this i have added another invokehttp with post command and url like this :https://server ip/nifi-api/flowfile-queues/{id}/drop-requests ( p.s i want to clear queue from flowfile) but first invokehttp shows me exceptions like this :Socket timeout exception and there are several subject i am interested in :
Is it possible that my user doesn't have access to berer token and that's why socket timeout happens? if it is how can i checke this.
if i have secured nifi is it possible to use invokehttp processor without ssl context service to fulfill this accsess/token operation?
Do you have any better idea to rest api without bearer token
except using ssl context service?
socket timeout is a physical error not linked to auth
when auth is activated for nifi then you must use ssl context to access nifi-api
no. if auth activated on server side then client have to pass it.
I need to patch Wildfly10 using HTTP or native management API. I know how to do it with CLI using command
patch apply /home/user/patch.zip
but is it possible to apply patch using HTTP or native management API?
Also will that patch be applied to all servers in the targeted server-group?
It is possible. It can be done using native management API.
https://github.com/wildfly/wildfly-core/blob/master/patching/src/main/java/org/jboss/as/patching/tool/PatchOperationTarget.java#L384
It can also be done calling /management-upload url using multipart. One part of multipart is patch zip file, and the second part is json holding request for CLI command /core-service=patching:patch(input-stream-index=0)
you can view this behavior by patching server using admin console and looking in web browsers console network