I'm calling the IBM Bluemix Administration API for Message Hub (aka Kafka), as described here.
Calling the endpoint https://kafka-admin-prod02.messagehub.services.eu-gb.bluemix.net:443/topics (using an appropriate X-Auth-Token value corresponding to our API key, and a GET verb) seems to work - it returns a list of the topics we've manually configured in the Bluemix Message Hub admin screen.
However, calling the same URL with the verb POST and a body of {name: 'mynewtopicname'} and a Content-Type of application/json (this appears to be the correct syntax, according to the Swagger docs for the API) doesn't work - I get a result of "HTTP 405 Method Not Allowed". That seems to happen whether I use GET, PUT, POST or DELETE (which also means I cannot delete a topic).
Am I doing something wrong?
(I'm using Message Hub on the UK Bluemix instance, if it's relevant).
To create topics you need to do a POST to the URL /admin/topics, not /topics. Give it a try and let me know if it works.
Related
I have built an AWS API Gateway Rest API integration with AWS SNS.
(Setup in simplified form)
Post Requests --> RestAPI in AWS API Gateway -->SNS
I want to configure the Gateway to respond to the post request with one field I get from PostRequestsBody.
But in the docs that I have read, I am able to send just the response body of the SNS request.
1>Is it possible to respond with body PostRequest as a response body in Gateway?
2> if yes please direct me to the docs or please explain how it can be achieved.
I have gone through http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html
which had explained just how to respond with details of integrated service(SNS in this case) response.
Even though this question is old now, it appears in search results.
AWS doesn't directly have a way to do this, but people have found hacks such as https://stackoverflow.com/a/61482410/563812
The short story: If I hit my endpoint /api/something with a PUT and no body, it goes through to my Lambda (via CF -> API GW -> Lambda). If I add a body to the request, it doesn't work and seems to die before hitting the API Gateway. I'm not sure why it would do this.
The long story:
I've set up a CloudFront distribution that sends requests to /api to our API gateway and all other requests just vend static website resources.
It has been working great so far for our simple use cases. We previously only hit the API Gateway with GET requests, but now we're going to start needing to send PUT or POST requests to the API Gateway. I had to update the CloudFront distribution "behaviors" for the /api path pattern (the one going to the API gateway) to allow all HTTP methods after that, I was able to start getting responses from the API Gateway for PUT and POST responses (previously only worked for GET).
However, I noticed that if I try to send a payload or body with the request (regardless of the request type), I don't think it actually hits the API Gateway or the Lambda that the API Gateway is routing to because I get a response of "You need to enable JavaScript to run this app". I tried looking at the logs but couldn't find anything. The heads shows "Error from cloudfront" but I'm not sure why or where to find what the error was.
Any tips on how to troubleshoot this or what the issue might be?
Update (10/8): I figured out the "enable javascript" thing was coming up because of my custom error pages for 403&404 errors. I deleted them in my development environment and now get a clearer error.
"This distribution is not configured to allow the HTTP request method
that was used for this request. The distribution supports only
cachable requests. We can't connect to the server for this app or
website at this time. There might be too much traffic or a
configuration error. Try again later, or contact the app or website
owner."
However, I still don't know what to do, as the request type (PUT) is enabled and the request works as long as I don't provide a body. Not sure how to get it to be ok with a body at the moment.
I don't mind if you use an example from another API that is not Adobe Analytics'. I just need to know the pattern that I have to follow in order to succesfully convert a Postman request into a NiFi request.
After successfully creating requests to pull reports from Adobe Analytics via Postman, I´m having difficulties to migrate these Postman requests to NiFi. I haven´t been able to find concrete use cases that explicity explain how to do this kind of task step-by-step.
I'm trying to build a backend on top of NiFi to handle multiple data extracts from Adobe Analytics in an efficient and robust way. That is instead of having to create all required scripts by myself. Yet, there is more documentation about REST APIs and Postman cases than there is about REST APIs and NiFi cases.
In the screenshot below we can see how the Postman request looks like. It takes 3 headers and 1 temporary header that includes the authorization value (Bearer token). This temporary header is generated automatically after filling in the OAuth 2.0 authorization form in the Authorization tab, as shown here.
Then, we have the body of the request. This json text is generated automatically by debugging Adobe Analytics' workspaces as shown here.
I'd like to know the following in a step-by-step manner with screenshots if possible:
Which processor(s) should I use in NiFi to obtain a similar response as the one I got in Postman?
Which properties should I add/remove from the processor to make this work?
How should I name these properties?
Is there a default property whose value/name I should modify?
As you can see, the question mainly refers to properties setup in NiFi, as well as Processor selection. I already tried to configure some processors but I don't seem to get the correct properties setup, or maybe I'm selecting the wrong processors.
I'm using NiFi v1.6.0 and Postman v7.8.0
This is most likely an easy task for users already familiar with NiFi and API requests, but it has proven challenging to me. Hopefully this will help other users looking to build more robust pipelines by using NiFi instead of doing it manually.
Thanks.
It only takes 3 NiFi processors to replicate a REST API request that works in Postman. In this solution we use a request that contains a nested JSON request. The advantage of this simple approach is that it reduces the amount of configuration required to obtain a successful response from the API. That is, even if you are using a complex JSON request. In this case the body of the JSON request is passed through the GenerateFlowFile processor, without the need of any other processor to parse/format the request.
Step #1. Create a processor called GenerateFlowFile. The only property that you will have to modify is the Custom Text. Paste in there your whole JSON request just as it was in Postman. In this case I'm using the very same JSON shown in the question above. It's a good idea to setup Yield Duration to 10 seconds or more.
Step #2. Create a processor called InvokeHTTP. Then modify the 6 properties shown in the screenshots below. Use the same Authorization details you've used in Postman. Make sure to copy the Bearer token from Postman after it has been tested. Also, don't forget to setup the HTTP Method, Remote URL and Content-Type as well.
Step #3. Finally, add a couple of LogAttribute processors to store the output of InvokeHTTP. One of these LogAttribute processors should store successful responses. The other one can be used for Failure, Original, Retry and No-Retry. Or you can create LogAttribute for each of these outputs.
Step #4. Now, connect the processors and Start your data flow! You should start seeing data populate the Successful LogAttribute. Then you can use the Data Provenance option to review the incoming data and confirm that this is exactly the same result you previously obtained from Postman.
Note: This is a simple, straightforward, "for starters" solution to replicate a Postman API request using a nested static JSON. There are more solutions in StackOverflow that tackle more complex cases, like dynamic JSON. Here's a list of some other posts:
nifi invokehttp post complex json
In NiFi processor 'InvokeHTTP' where do you write body of POST request?
Configuring HTTP POST request from Nifi
I want to use hub to create some automation script but can't figure out how to post a message to the api. More specifically it is this api endpoint I want to post to: https://developer.github.com/v3/users/gpg_keys/#create-a-gpg-key
I've tried the following:
hub api user/gpg_keys -F armored_public_key='-----BEGING PGP ....'
hub api user/gpg_keys --raw-field "body={armored_public_key='-----BEGING PGP ....'}
None of the above work, the response back is:
{"message":"Not Found","documentation_url":"https://developer.github.com/v3/users/gpg_keys/#create-a-gpg-key"}
which is not very helpful. I thought that -F meant fields within the payload, but I guess I'm wrong.
So how do I post a message with a body using hub api?
UPDATE
I did download the repo to create a modified version that has some more logging, but can't figure out what's wrong. Everything looks good from what I understand in how the request is made.
Looking through the code I found the reason to why this is. It seems like it is hard coded that you should only get the repo scopes when you log in the first time. If you use a token you've created yourself that has more access everything will work as it should.
I am using the DocuSign REST API to send documents and we are trying to take advantage of the webhook capabilities.
Our problem is that we have basic auth set up on our webhook listener which it doesn't appear is supported in the REST API. I am enquiring whether or not this is an accurate understanding. I know it is possible on the Connect product, which the basic auth can be set up via the portal, however I am looking specifically for API REST calls.
How can I secure the EventNotification event?
Many thanks,
Mitch.
As of now, according to docusign offical post:
Individual Envelope Connect configurations created with the eventNotifications API option do not support Basic Authentication at this time.
A WORKAROUND SOLUTION:
Docusign has an official blog post to answer this question. Securing Your Connect Webhook Listener
For us, we use the Use a Pre-shared Secret method to secure our webhook. We are using the python sdk. During the process we create the EventNotification, we add a secret parameter to the webhook listener url. When the docusign post the request to your listener server, it will include the secret. You could check the parameter to make sure the request is valid.
event_notification = EventNotification()
event_notification.url ='your webhook_url' + '?secret=' + 'your_secret')
The text below is quoted from the blog post.
This defense acts both as access control and authentication. The
listener URL you provide to DocuSign can include one or more query
parameters. DocuSign will include them during its POST request to your
listener.
For access control, your listener will first check that the request
includes the expected query parameter and reject all requests that
don’t. For authentication, your listener will additionally check the
value of the query parameter. Remember that you can encode any values
for the name and value of the query parameter. For this example, we’re
using “pw” as the name of the query parameter.
To use a pre-shared secret, just set the URL accordingly in the
Connect configuration. For example,
https://listener_url.example.com/listener?pw=secret
Remember that the complete URL, including its query parameters, is
encrypted before it is sent across the internet. The URL and its query
parameters are visible in various logs and configuration screens,
including the Connect webhook configuration page.