I'm trying to figure out where to pull resourceId of an apigateway using this documentation - https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/apigateway.html
I was able to retrieve restApiId using the method get_base_path_mapping(). There are other boto3 API-Gateway methods that I need to use but it requires resourceId as a required parameter. However, I can't find which method returns the resourceId value. I checked get_resource() method but looks like it's not the one since it also requires resourceId.
get_resources will do the trick
Related
Using Argument setter by supplying the parameter value I want to make the Datafusion pipeline as resuable. As said by many other answer's have tried implementing using the cloud reusable pipeline example given in Google guide.I was not able to pass the parameter Json file.So how to create the API to that parameter Json file stored in Google storage.Please explain the values to be passed to Argument setter like URL,Request response etc., If any one of you had implemented in your projects.
Thank you.
ArgumentSetter plugin reads from a HTTP endpoint and it must be publicly accessible as is depicted within the GCP documentation. Currently, there is not a way to read from a non-public file stored in GCS. This behavior has been reported in order to be improved to CDAP through this ticket.
Can you please provide what you've tried so far and where you're stuck?
The URL field in argument setter would contain the API endpoint you're making a call to. Make sure you include any headers your call would need like Authorization, Accept etc.
If you're having issues with argument setter a good check is to use Curl or any other tool to make sure you're able to talk to the endpoint you're trying to use.
Here's some documentation about Argument setter: https://github.com/data-integrations/argument-setter
Define a JSON file with appropriate name/value pairs. Upload it in a GCS bucket - make it public by changing permissions (add "allUsers" in permissions list). When you save it, the file will say "Public to Internet"
Copy the https path to the file and use it in Arguments Setter. If you're able to access this path from curl/ your browser, Argument Setter will be able to do too..
There are other problems I've encountered while using Argument Setter though - the pipe doesn't supersede runtime arguments over default values provided in the URL many a times, specially when the pipe is duplicated.
To make file public
You have to make your bucket public, currently there is no other way.
gsutil iam ch allUsers:objectViewer gs://BUCKET_NAME
For the alpha-vantage python module, does anyone know what the argument treat_info_as_error does?
This is what I found in terms of documentation. It's not much.
TimeSeries
Definition : TimeSeries(key=None, output_format='json', treat_info_as_error=True, indexing_type='date', >proxy=None)
This class implements all the api calls to times series
When you run an almost invalid API call on Alpha Vantage you'll get a key in the JSON response that says "note" or "Information" or something like that. This parameter is whether or not you want to treat those in your code as an error.
For some reference, here is a link to the docs.
For example, if you hit an API call threshold, or use the "demo" API key for calls other than what they are suited for.
Try this API call:
https://www.alphavantage.co/query?function=TIME_SERIES_INTRADAY&symbol=TSLA&interval=5min&apikey=demo
You'll see it says "Information" as the first key.
Using openapi-generator to generate a C# API client using an openapi v3.0 specification. Is there a way to customize the methodNames of the API? I'd prefer to be able to specify them maybe by using operationId field?
Right now they are incorporating the path variables along with the HTTP VERB and the static path components. It'll work, but I'd like the names to be a bit different.
GetEncountersResponse ClientidEncountersDateGet (string clientid, string date, string startToken = null);
The other option is to just add new methods since the class is defined as partial?
Right now they are incorporating the path variables along with the HTTP VERB and the static path components. It'll work, but I'd like the names to be a bit different.
Looks like the operationId (optional) is missing in the operation (endpoint) defined in the OpenAPI doc/spec and that's why the method name is auto-generated based on the path, HTTP verb, etc.
You can define the operationId to have better control of the method name.
I could not understand what the package can do, the offical doc show nothing about unstructured. What the package used for ? Is it used for converting map[string]interface{} to K8S Obj ?
https://godoc.org/k8s.io/apimachinery/pkg/apis/meta/v1/unstructured
It looks like unstructured provides an interface to kubernetes objects, when you don't know object type upfront, i.e. dynamic package in client-go uses it extensively
As #kseniia-churiumova suggested it is used when you don't know the object type. Here is the use case to understand it better. Let us say your organisation has a policy that all Kubernetes object must have annotation "owner" with value pointing to email ID of a person or group. You have tasked with finding all the resources that violates this policy.
You can have a configuration file that has list of GroupVersionKind and use unstructured to query them. If a new type needs a check you can add to configuration without changing the code.
Note: This is just an example. In production you will have to use something like Gatekeeper that implements OPA specification to enforce policies.
I would like to modify an url parameter /resource/{VaRiAbLe} in an API gateway to S3 mapping so that it actually points to /my-bucket/{variable}. That is, it accepts mixed-case input, and maps it to a lower-case name. Mapping path variables is relatively simple enough to S3 integrations, but I can't seem to get a lower-case mapping working.
Reading through the documentation for mapping parameters, it looks like the path parameters are simple string values (and not templated values), and so defining a mapping as method.request.path.variable.toLowerCase() won't work.
Does anyone have any ideas how to implement this mapping?
Map path variables to a JSON body, and then call another API method that actually does the S3 call?
Bite the bullet, and implement a Lambda function to do the S3 get for me?
Find another api method for S3 that accepts a JSON body that I can use to get the data?
Update using Orchestrated calls
Following the info from Jack, I figured I should try doing the orchestrated call, since the traffic volume is low enough that I am sure that I won't be able to keep the lambda hot.
As a proof of concept, I added two methods to my resource (sitting at /resource/{variable} - GET and POST. The GET method chains to the POST, which does the actual retrieving of the data.
POST method configuration
This is a vanilla S3 proxying method, where you set the URL Path parameter for {variable} to be method.request.body.variable.
GET method configuration
This is a HTTPS proxying method. You'll need an URL for the POST method, so you'll need to deploy the API to get the URL. The only other configuration needed here is a body mapping template with content like:
{
"variable" : "$input.params('variable').toLowerCase()",
"something" : "$input.params('something')"
}
This should be enough to get this working.
The downside to this looks to be that I'm adding an extra method (POST) to my API for that resource that could confuse consumers of the API. I think it should be possible to make the POST on the /resource resource, which would at least make a bit more sense from an API design standpoint.
Depending on how frequently this API will be called, I'd either go with the Lambda proxy or chaining two API Gateway methods together. If the API is called frequently enough to keep a Lambda function warm (say once a minute), then go with Lambda. If not, go with the orchestrated API call.
The orchestrated API call would be interesting, I'd be happy to help with that if you have questions.
As far as I know the only S3 API for getting object data is the GET that is documented in their API reference.