Our service implements different levels of access and we are using one openAPI YAML file internally.
For external documentation purposes, we would like to create multiple openAPI files, that are valid in themselves (self-sustained), but only have a partial set of the global file, e.g. based on the path or on tags.
(The same path may be used in different split-Files but I don't think that is a problem then.)
Any idea on how to achieve that? Is there some tooling around for it?
You can use a valid URI in a JSON Pointer which points to another resource. The URI can be a path to a local file, a web resource, etc.:
paths:
/user/{id}:
summary: Get a user
parameters:
- $ref: "./path/to/file#/user_id"
# And so on...
Reserved keys in the OpenAPI spec must be unique so I don't think you'd be able to create standalone OpenAPI specs without some third-party utility that could overcome that limitation.
However, you would be able to create valid standalone JSON objects defined across many files and reference them in the index document. There are many articles online providing examples:
https://davidgarcia.dev/posts/how-to-split-open-api-spec-into-multiple-files/
https://blog.pchudzik.com/202004/open-api-and-external-ref/
I ended up writing a Python script, that I have posted here.
Flow
Read the YAML File into a dictionary
Copy the dictionary to a new dictionary
Iterate through the original dictionary and
Remove items that are not tagged with the tag(s) you want to keep
Remove items that are have some keyword you want to omit in the path
Write out the dictionary to a new YAML
The GIST is available here:
https://gist.github.com/erikm30/d1f7e1cea3f18ece207ccdcf9f12354e
Related
It is straightforward to put resource id into url if it is a int or long type. e.g.
GET files/123
But my problem is that my resource identifier is a path. e.g. /folder_1/folder_2/a.sh because the underlying implementation is a filesystem. So I can not put it as part of rest api url because it is conflict with url path.
Here's approaches what I can think of:
Put the path id as the request param. e.g.
GET files?path=/folder_1/folder_2/a.sh
Encode/decode the path to make it qualifier as part of url.
Introduce another int/long id for this resource in backend. And map it to the path. The int/long type resource id is stored in database. And I need to maintain the mapping for each CURD operation.
I am not sure whether approach 1 is restful, approach 2 needs extra encoding/decoding, and approach 3 needs extra work to maintain the mapping.
I wonder what is the best practice to design the rest api url for this kind of case.
Simple:
#GET
#Path("/files/{path:.+}")
#Produces({MediaType.TEXT_PLAIN})
public String files(
#PathParam("path") String path
) {
return path;
}
When you query files/test1/tes2 via url output is:
test1/tes2
Just put the path after a prefix, for example:
GET /files/folder_1/folder_2/a.sh
There isn't a conflict, since when the request path starts with your known prefix (/files/, in the above example), you know that the rest should be parsed as the path to the file, including any slashes.
Well, my experience designing "restful" APIs shows that you have to take into consideration future extensions of your API.
So, the guidelines work best when followed closely when it makes sense.
In your specific example, the path of the file is more of an attribute of the file, that can also serve as its unique ID.
From your API client's perspective, /files/123 would make perfect sense, but /files/dir1/file2.txt is debatable.
A query parameter here would probably help more, much like what you would do if you wanted to retrieve a filtered list of files, rather than the whole collection.
On the other hand, using a query parameter would also help for future extensions, since supporting /files/{path} would also mean conflicts when attempting to add sub-resources to your files endpoint.
For example, let's assume that you might need in the future another endpoint /files/attributes. But, having such an endpoint, would exclude any possibility for your clients to match a file named attributes.
Using Argument setter by supplying the parameter value I want to make the Datafusion pipeline as resuable. As said by many other answer's have tried implementing using the cloud reusable pipeline example given in Google guide.I was not able to pass the parameter Json file.So how to create the API to that parameter Json file stored in Google storage.Please explain the values to be passed to Argument setter like URL,Request response etc., If any one of you had implemented in your projects.
Thank you.
ArgumentSetter plugin reads from a HTTP endpoint and it must be publicly accessible as is depicted within the GCP documentation. Currently, there is not a way to read from a non-public file stored in GCS. This behavior has been reported in order to be improved to CDAP through this ticket.
Can you please provide what you've tried so far and where you're stuck?
The URL field in argument setter would contain the API endpoint you're making a call to. Make sure you include any headers your call would need like Authorization, Accept etc.
If you're having issues with argument setter a good check is to use Curl or any other tool to make sure you're able to talk to the endpoint you're trying to use.
Here's some documentation about Argument setter: https://github.com/data-integrations/argument-setter
Define a JSON file with appropriate name/value pairs. Upload it in a GCS bucket - make it public by changing permissions (add "allUsers" in permissions list). When you save it, the file will say "Public to Internet"
Copy the https path to the file and use it in Arguments Setter. If you're able to access this path from curl/ your browser, Argument Setter will be able to do too..
There are other problems I've encountered while using Argument Setter though - the pipe doesn't supersede runtime arguments over default values provided in the URL many a times, specially when the pipe is duplicated.
To make file public
You have to make your bucket public, currently there is no other way.
gsutil iam ch allUsers:objectViewer gs://BUCKET_NAME
I have a really complicated system which use multiple languages and frameworks (Java Python Scala Bash). In each module I need to retrieve configuration values which are similar and change frequently. Currently I'm maintaining multiple conf files which holds lots of duplicates.
I wonder if there is out of the box RestAPI which can retrieve variables by demand from remote location.
All I manage to find by now are ways to load the entire file from remote source which is half a solution from me:
YAML.parse(open('https://link_to_file/file.yaml'))
My goal, which I fail to find a lead to it, is to make a direct call.
MyRemoteAPI.get("level1.level2.x")
P.S
YAML is not mandatory solution for me, I'm Open for suggestions.
I don't know about an out-of-the-box API, but it's fairly trivial to build. Make a service that will read the YAML file and traverse to the appropriate key. e.g. using a dynamic language like Ruby (+Rails), you could do something like
def value
config = YAML.load_file '/local/path/to/config.yaml'
render plain: config.dig(params[:key].split('.'))
end
dig essentially traverses a structure and safely returns nil if a key isn't found, so this returns the value at the "leaf" of the requested path.
You might also want to cache the structure in memory to prevent constantly reading from the file, e.g. could do something like ##config ||= YAML.parse(open('https://link_to_file/file.yaml')) or config = Rails.cache.fetch('config', expire_in: 1.hour) { ... }. And/or cache the API's HTTP response.
I'm defining my own Puppet class, and I was wondering if it is possible to have an array variable which contains a list of all files in a specific directory. I was wondering to have a similar syntax like below, but didn't found a way to make it work.
$dirs = Dir.entries('C:\\Program Files\\Java\\')
Does anyone how to do it in a Puppet file?
Thanks!
I was wondering if it is possible to have an array variable which contains a list of all files in a specific directory.
Information about the current state of the machine to be configured is conveyed to the catalog compiler via facts. These are available to your classes as top-scope variables, and Puppet (or Facter, actually) provides ways to define your own custom facts. That's a link into the Facter 3 manual, but similar applies to earlier versions. Do not overlook the rest of the Facter documentation, which has more relevant information on this topic.
On the other hand, information about the machine providing catalog-building services -- the master in a master / agent setup -- can be obtained by writing and calling a custom function. This is rarely what you actually want, but it's worth mentioning because you might one day want a custom function for some other purpose.
I use a xtext generated DSL in my project. I now want to generate some files with a wizard. I can currently create these files just by adding some strings to the file. But initially i wanted to create a Model Object of the DSL and add the new generated file to the resource set. I can't find a way accessing this without the StandaloneSetup of the DSL.
I'm now unsure if the use of the StandaloneSetup inside the Editor is the correct way, since it seems there should be a better way to access the already build up resource set used by the xtext framework to manage the already known files/sources. If i use the StandaloneSetup shouldn't there be 2 Injectors which could get really bad?
you can use the resource service provider registry to obtain the injector
http://koehnlein.blogspot.de/2012/11/xtext-tip-how-do-i-get-guice-injector.html
URI fakeOrRealUri = ...;
IResourceServiceProvider.Registry.INSTANCE.getResourceServiceProvider(fakeOrRealUri).get()