Versioning of REST webservices on top of gRPC - rest

I've implemented an API service using gRPC with protocol buffers and then used grpc-gateway to expose that as a set of REST webservices.
Now I'm getting to the point where I'm having to maintain different versions of the API and I'm stuck.
In my proto file I have a handler like this defined for instance
rpc MerchantGet (MerchantRequest) returns (MerchantResponse) {
option (google.api.http) = {
get: "/v1.1.0/myapi/merchant/{MerchantID}"
};
}
In my Go code of course I then have a function, MerchantGet, to which GET actions to /v1.1.0/myapi/merchant/{MerchantID} are mapped.
Now, let's say I want to add more functionality to the MerchantGet method and release a new version. I intend to maintain backwards compatibility as per the Semantic Versioning Specification so if I understand correctly that means I can make underlying changes to my MerchantGet method and have it supersede the older method as long as it does not require different inputs from the 3rd party (MerchantRequest) or change the response sent to the 3rd party (MerchantResponse) other than by adding additional fields to the end of the response. (Correct me if I'm wrong in this assumption).
My question is, how do I write proto handlers to serve a method to endpoints of different versions? One option that came to mind would look something as follows:
rpc MerchantGet (MerchantRequest) returns (MerchantResponse) {
option (google.api.http) = {
get: "/v1.6.0/myapi/merchant/{MerchantID}"
additional_bindings {
get: "/v1.5.0/myapi/merchant/{MerchantID}"
}
additional_bindings {
get: "/v1.4.2/myapi/merchant/{MerchantID}"
}
additional_bindings {
get: "/v1.4.1/myapi/merchant/{MerchantID}"
}
additional_bindings {
get: "/v1.4.0/myapi/merchant/{MerchantID}"
}
additional_bindings {
get: "/v1.3.0/myapi/merchant/{MerchantID}"
}
additional_bindings {
get: "/v1.2.0/myapi/merchant/{MerchantID}"
}
additional_bindings {
get: "/v1.1.0/myapi/merchant/{MerchantID}"
}
};
}
But surely this can't be the idiomatic way of achieving this? It's certainly not very elegant at all as, with each new minor version or patch, I would have to extend these additional_bindings to each of my methods (above I'm just using one method as an example).

From the SemVer spec:
Given a version number MAJOR.MINOR.PATCH, increment the:
MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards-compatible manner, and
PATCH version when you make backwards-compatible bug fixes.
Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
The only version that matters with respect to REST endpoint versioning is the MAJOR version, because all MINOR and PATCH changes must be backwards-compatible.
To answer your question:
Use only major version numbers in the REST URI. The rest is an implementation detail, from a REST standpoint.
So, your proto service will be:
rpc MerchantGet (MerchantRequest) returns (MerchantResponse) {
option (google.api.http) = {
get: "/v1/myapi/merchant/{MerchantID}"
};
}

Related

Best practice for including common http error codes in swagger / openapi definition

I'm wondering about good practices about including common error types in Swagger/OpenAPI definition.
Let's consider following controller method:
[HttpGet]
[ProducesResponseType(StatusCodes.Status400BadRequest)] // to be or not to be?
public ActionResult SomeMethod(Foo foo)
{
if (foo.Property != "expectedValue")
{
return BadRequest();
}
return Ok();
}
So, I'm performing some logic in the controller, which might end up in a state in which I want to return 400 BadRequest. Note, that I don't return any content.
Since I develop a REST API that generates Swagger/OpenAPI definition and tools like autorest may be used to generate client code based on that definition, I want to be sure it is as accurate as possible.
My question is:
should I declare explicitly that the user might get 400 Bad Request
OR
this makes sense only in case I want to describe the format of response content (in case 400Bad request has some content)
The same applies to codes like 401, 404 etc.
Even if (in addition to the returned http code, 400, 401, etc..) you do not return any payload at all, you should yet explicitly declare such responses.
Do not forget that in a "code-first" approach, your build chain could automatically generate the OpenApi contract (based on your code) as well as a nice user documentation.
In the contrary approach (contract-first), the code generated based on your OpenApi file would contain these response descriptions.
If you plan to use client code generation, then it's always better to use these declarations, because code generators (like autorest or NSwag) can use this information (usually from a swagger definition aka swagger.json) to generate more correct responses and error handling procedures.
Here, for example, you can see what can happen if you omit such declarations.
Also, remember, that in API controllers you can use web API conventions.
It says that these conventions allow you to:
Define the most common return types and status codes returned from a specific type of action.
Identify actions that deviate from the defined standard.
So, generally speaking, they can help you to make your code more concise.
The object BadRequest is used as a response (Result) with StatusCode=400.
DotNet has several response types with predefined StatusCode, so you can use different methods, for example:
NotFound(404):https://learn.microsoft.com/en-us/dotnet/api/system.web.http.apicontroller.notfound?view=aspnetcore-2.2
InternalServerError(500):https://learn.microsoft.com/en-us/dotnet/api/system.web.http.apicontroller.internalservererror?view=aspnetcore-2.2
Or simply use StatusCode for any other code that is not predefined:https://learn.microsoft.com/en-us/dotnet/api/system.web.http.apicontroller.statuscode?view=aspnetcore-2.2

Are there disadvantages to splitting a class's methods across different extensions?

I'm writing an API service (called ApiService) that handles calls to a server which create, read, update and delete many different types of data. Each type of data has its own set of calls e.g. postsList, postCreate, postRead, postUpdate and postDelete.
If there are say 7 different types of data each with at least 5 API calls then the ApiService class becomes large and cumbersome to work with (say >3000 lines). I want to find a way to segment the API service so it can be worked on more easily, but still allow for my app to find any given API call it needs to through only the base API service, rather than having to pull in a PostsApiService or CommentsApiService when it needs to make a particular API call.
A way to I've thought of which achieves what I'm looking is through extensions. I can keep each extension in its own file and use the part of and part directives so the extensions are included in the base file containing the original API service and when ApiService is used across the app I can access all the extension methods:
// post_api.dart
part of 'api_service.dart';
extension PostApi on ApiService {
Future<List<Posts>> posts() {
...
}
Future<List<Posts>> postCreate() {
...
}
...
}
// api_service.dart
part 'post_api.dart';
class ApiService {
...
}
I quite like this approach, it helps me reason about the different aspects of the Api much more easily than before but it does feel quite hacky. Additionally since the base ApiService doesn't have any methods of its own, I feel like I'm misusing darts extension functionality.
Does this way open me up to all sorts of unforeseen complications, performance issues or bad practises? Is there a better way to achieve the segmentation I'm looking for?

config=false node when mandatory

I have a Yang model which defines a config=false node which is mandatory too. Should I return that node as an empty XML node in get rpc response, even when my app does not support it?
Ideally my app is supposed to support it, but due to limitations we cannot implement the required support. So what should be the right way to handle such cases? Should we emit/represent it in get rpc response as an empty XML node? I guess if we ignore such nodes, the external controller may fail the get rpc response.
-Ram
If your server implementation does not support a specific node in the original model, you should create a deviation YANG module, which expresses this limitation. This way clients are informed about it and everyone is happy - you of course advertise your deviation module along with the deviated one.
For example:
module target {
yang-version 1.1;
namespace "target:uri";
prefix "tgt";
container state {
config false;
leaf some-counter {
type uint64;
mandatory true;
}
}
}
Let's say your device cannot support some-counter leaf above. You then create create a deviation module, which describes how your implementation differs from a compliant implementation.
module target-dev {
yang-version 1.1;
namespace "target-dev:uri";
prefix "tgtd";
import target {
prefix tgt;
}
deviation "/tgt:state/tgt:some-counter" {
deviate not-supported;
}
}
When a get request comes, you return nothing for that leaf, since it does not exist in your implementation's world.
The details of deviation and deviate statements may be found in RFC7950:
https://www.rfc-editor.org/rfc/rfc7950#section-5.6.3
https://www.rfc-editor.org/rfc/rfc7950#section-7.20.3
https://www.rfc-editor.org/rfc/rfc7950#section-7.20.3.2
You should be very careful when relying on this mechanism! Always create a separate module, which contains only the deviations, possibly deviating a single target module. There is a guidelines document you should read just in case.

Retrofit2 - How to execute a JSON Patch Request (RFC 6902)

As of Retrofit2 2.3.0 there seems to be no built in functionality to execute a JSON Patch Request (as defined in RFC 6902. Also, see http://jsonpatch.com/ for some examples).
Using the available #PATCH annotation, the full blown object is sent with the request (as if I would send a PUT request, which is not what I'm looking for)
public interface MyService {
#PATCH("example/{id}")
Call<Example> patchExample(#Path("id") String id, #Body Example example);
}
After a first glance at the Retrofit documentation, there seems to be no clean and easy way to introduce a custom annotation (e.g. #JSONPATCH) to have my own implementation working.
The only related information I was able to find regarding this requirement was this experimental (as he calls it himself --> this is very experimental but it does the job currently) approach at https://medium.com/#andretietz/custom-annotations-with-retrofit-2-8701ca7ce102. I didn't give this example a try, but the complexity seems a little bit out of scale for this simple requirement.
Maybe I'm missing something and there is an easy solution for this?

How to version REST URIs

What is the best way to version REST URIs? Currently we have a version # in the URI itself, ie.
http://example.com/users/v4/1234/
for version 4 of this representation.
Does the version belong in the queryString? ie.
http://example.com/users/1234?version=4
Or is versioning best accomplished another way?
Do not version URLs, because ...
you break permalinks
The url changes will spread like a disease through your interface. What do you do with representations that have not changed but point to the representation that has? If you change the url, you break old clients. If you leave the url, your new clients may not work.
Versioning media types is a much more flexible solution.
Assuming that your resource is returning some variant of application/vnd.yourcompany.user+xml all you need to do is create support for a new application/vnd.yourcompany.userV2+xml media type and through the magic of content negotiation your v1 and v2 clients can co-exist peacefully.
In a RESTful interface, the closest thing you have to a contract is the definition of the media-types that are exchanged between the client and the server.
The URLs that the client uses to interact with the server should be provided by the server embedded in previously retrieved representations. The only URL that needs to be known by the client is the root URL of the interface. Adding version numbers to urls only has value if you construct urls on the client, which you are not suppose to do with a RESTful interface.
If you need to make a change to your media-types that will break your existing clients then create a new one and leave your urls alone!
And for those readers currently saying that this makes no sense if I am using application/xml and application/json as media-types. How are we supposed to version those? You're not. Those media-types are pretty much useless to a RESTful interface unless you parse them using code-download, at which point versioning is a moot point.
I would say making it part of the URI itself (option 1) is best because v4 identifies a different resource than v3. Query parameters like in your second option can be best used to pass-in additional (query) info related to the request, rather than the resource.
Ah, I'm putting my old grumpy hat on again.
From a ReST perspective, it doesn't matter at all. Not a sausage.
The client receives a URI it wants to follow, and treats it as an opaque string. Put whatever you want in it, the client has no knowledge of such a thing as a version identifier on it.
What the client knows is that it can process the media type, and I'll advise to follow Darrel's advice. Also I personally feel that needing to change the format used in a restful architecture 4 times should bring huge massive warning signs that you're doing something seriously wrong, and completely bypassing the need to design your media type for change resiliance.
But either way, the client can only process a document with a format it can understand, and follow links in it. It should know about the link relationships (the transitions). So what's in the URI is completely irrelevant.
I personally would vote for http://localhost/3f3405d5-5984-4683-bf26-aca186d21c04
A perfectly valid identifier that will prevent any further client developer or person touching the system to question if one should put v4 at the beginning or at the end of a URI (and I suggest that, from the server perspective, you shouldn't have 4 versions, but 4 media types).
You should NOT put the version in the URL, you should put the version in the Accept Header of the request - see my post on this thread:
Best practices for API versioning?
If you start sticking versions in the URL you end up with silly URLs like this:
http://company.com/api/v3.0/customer/123/v2.0/orders/4321/
And there are a bunch of other problems that creep in as well - see my blog:
http://thereisnorightway.blogspot.com/2011/02/versioning-and-types-in-resthttp-api.html
These (less-specific) SO questions about REST API versioning may be helpful:
Versioning RESTful services?
Best practices for web service REST API versioning
There are 4 different approaches to versioning the API:
Adding version to the URI path:
http://example.com/api/v1/foo
http://example.com/api/v2/foo
When you have breaking change, you must increment the version like: v1, v2, v3...
You can implement a controller in you code like this:
#RestController
public class FooVersioningController {
#GetMapping("v1/foo")
public FooV1 fooV1() {
return new FooV1("firstname lastname");
}
#GetMapping("v2/foo")
public FooV2 fooV2() {
return new FooV2(new Name("firstname", "lastname"));
}
Request parameter versioning:
http://example.com/api/v2/foo/param?version=1
http://example.com/api/v2/foo/param?version=2
The version parameter can be optional or required depending on how you want the API to be used.
The implementation can be similar to this:
#GetMapping(value = "/foo/param", params = "version=1")
public FooV1 paramV1() {
return new FooV1("firstname lastname");
}
#GetMapping(value = "/foo/param", params = "version=2")
public FooV2 paramV2() {
return new FooV2(new Name("firstname", "lastname"));
}
Passing a custom header:
http://localhost:8080/foo/produces
With header:
headers[Accept=application/vnd.company.app-v1+json]
or:
headers[Accept=application/vnd.company.app-v2+json]
Largest advantage of this scheme is mostly semantics: You aren’t cluttering the URI with anything to do with the versioning.
Possible implementation:
#GetMapping(value = "/foo/produces", produces = "application/vnd.company.app-v1+json")
public FooV1 producesV1() {
return new FooV1("firstname lastname");
}
#GetMapping(value = "/foo/produces", produces = "application/vnd.company.app-v2+json")
public FooV2 producesV2() {
return new FooV2(new Name("firstname", "lastname"));
}
Changing Hostnames or using API Gateways:
Essentially, you’re moving the API from one hostname to another. You might even just call this building a new API to the same resources.
Also,you can do this using API Gateways.
I wanted to create versioned APIs and I found this article very useful:
http://blog.steveklabnik.com/posts/2011-07-03-nobody-understands-rest-or-http
There is a small section on "I want my API to be versioned". I found it simple and easy to understand. The crux is to use Accept field in the header to pass version information.
If the REST services require authentication before use, you could easily associate the API key/token with an API version and do the routing internally. To use a new version of the API, a new API key could be required, linked to that version.
Unfortunately, this solution only works for auth-based APIs. However, it does keep versions out of the URIs.
If you use URIs for versioning, then the version number should be in the URI of the API root, so every resource identifier can include it.
Technically a REST API does not break by URL changes (the result of the uniform interface constraint). It breaks only when the related semantics (for example an API specific RDF vocab) changes in a non backward compatible way (rare). Currently a lot of ppl do not use links for navigation (HATEOAS constraint) and vocabs to annotate their REST responses (self-descriptive message constraint) that's why their clients break.
Custom MIME types and MIME type versioning does not help, because putting the related metadata and the structure of the representation into a short string does not work. Ofc. the metadata and the structure will frequently change, and so the version number too...
So to answer your question the best way to annotate your requests and responses with vocabs (Hydra, linked data) and forget versioning or use it only by non backward compatible vocab changes (for example if you want to replace a vocab with another one).
I'd include the version as an optional value at the end of the URI. This could be a suffix like /V4 or a query parameter like you've described. You might even redirect the /V4 to the query parameter so you support both variations.
I vote up for doing this in mime type but not in URL.
But the reason is not the same as other guys.
I think the URL should be unique (excepting those redirects) for locating the unique resource.
So, if you accept /v2.0 in URLs, why it is not /ver2.0 or /v2/ or /v2.0.0? Or even -alpha and -beta? (then it totally becomes the concept of semver)
So, the version in mime type is more acceptable than the URL.