In an HTTP-driven microservices architecture, each service might have a number of public endpoints that return JSON, for example, to a client or an API gateway intermediary. These services could also accept POSTs with JSON bodies of a certain shape, or query strings of a certain shape, etc.
What are some good options for documenting or programmatically keeping track of these "contracts" between services? I.e, if service A's /getThing endpoint has been refactored to return different data, is there a documentation tool or methodology that would facilitate updating the API gateway to adapt to this change?
For programmatically management of contracts, if you using spring-cloud stack then you must look into spring-cloud-contract, by which you can easily keep track of your latest version of contracts for your Rest endpoints and also if any change occurs in your api endpoint, this will help you notify by breaking the contract and failing the test-cases build around it.
Let's say for example, service A's /getThing endpoint has been refactored to return different data then all calling services to this endpoint will fail while build time of your project.
However, this methodology won't facilitate updating the API gateway to adapt to this change as there might different logic you want to perform of every new version of your endpoints.
You can also create Rest Docs snippets using these endpoint contracts. checkout Rest Docs snippets. You can also use swagger for documenting your endpoints.
for NodeJs check here.
Related
I have an API, and a consumer web app, both written in Node and Express. The API is defined by a OpenAPI Specification. Implemented by swagger-ui-express.
The above web apps are Dockerised and managed in Kubernetes.
The API has a handful of endpoints for managing the lifecycle of a user's registration/application to the service.
Currently, when I need to cleardown completed/abandoned applications, or resubmit failed applications, I employ a periodically run cronjob to carry out a database query for the actions mentioned. The cronjob is defined by a Kubernetes config YAML file. This is quickly becoming unmanageable, and hard to maintain.
I am looking in to having a dedicated endpoint for each of the above tasks. Then a dedicated cronjob could periodically send a request to the API endpoint to carry out the complex task. This moves the business logic back in to the API, and avoids duplication within a cronjob hosted elsewhere. I am ultimately asking if this is a good approach or is there a better workflow documented somewhere I could implement?
My thinking is that I could add these new endpoints to the already-existing consumer API, but have the new (housekeeping/management) endpoints separated from the others.
To separate each (current) endpoint in to their respective resource, I am defining tags within the specification. Tags don't seem to be sufficient for the separation of these new "housekeeping" endpoints.
Looking through the SwaggerUI documentation I can see that I can define multiple definitions (via the urls property) to switch between. These definitions being powered by individual Specification documents. This looks like a very clean way of separating the consumer API from the admin API, is this best practice?
Any input would be appreciated on this as I am struggling to find much documentation on this kind of issue.
I want to mock api calls from my application, and host the mock, so my tests can work without calls to real api. There is a service called restbird which does exactly that, but it is far from ideal for me. If you want to collaborate you have to host the service by your self. Also it has some errors like not displaying history of calls, or when it sends server errors for no reason. I want a service more robust than this one.
The only service that I think might be a good fit is SwaggerHub, it seems robust, it has virtual servers, and overall it is very popular. But the only problem is that I cannot find a way to record api calls from my application. So how can I record api calls for SwaggerHub?
There does not currently exist any functionality within SwaggerHub itself to record API calls made from the Swagger UI module within the tool. This is a limitation of the open-source Swagger UI tool.
What I can recommend is you use the Swagger Inspector tool. The Swagger Inspector can be used to make API calls from a client, save both the request and the response, and even generate an OpenAPI file for you based off the request/responses. If you create an account and sign in, you can even save your API calls to a collection to use later.
Swagger Inspector: https://inspector.swagger.io/builder
It may also be worth considering using ReadyAPI's Virtualization module to handle this use case. With ReadyAPI Virtualization you can record transactions from a browser, build mock services from the recorded transaction or an existing API definition, and then host the mock service using VirtServer.
ReadyAPI is a part of SmartBears API lifecycle products, so there are integrations between the two tools. For instance, you can port APIs from Swaggerhub into ReadyAPI directly and you can use mock services built in ReadyAPI to do dynamic mocking in Swaggerhub.
You can find more information about ReadyAPI Virtualization here: https://smartbear.com/product/ready-api/api-virtualization/
I realise this is a very late response to this thread, but hopefully this information comes in handy.
I have an application API that is used In two scenarios:
My frontend application uses it to interact with the server
A client is using it for development of CLI tool so there is an open documentation of the API.
At start all of the endpoints were kind of generic so they have been used in both scenarios, but as my application grows i have a need to :
create special endpoints for my frontend application for optimization, for example an endpoint to some statistics screen
Change some of the basic API results structures that are not backward compatible and can break the Clients
usage.
What is the best practice to design an API to meet these needs?
How is should be design correctly so it will be adjusted
to the frontend needs and on the other side will be robust enough to not break the Client's applications?
frontend specific endpoints along with General ones?
What is the best practice to design an API to meet these needs?
This highly depends on your scenario. Is your API going to be used internally only or will it be made publicly available to an unknown number of developers and integrators? What is the expected lifetime of the API? Will it evolve?
How is should be design correctly so it will be adjusted to the frontend needs and on the other side will be robust enough to not break the Client's applications?
I recommend to commit to API contracts and use a specification for these contracts. I prefer the OpenAPI specification as it will come with a lof of benefits. Make sure you invest a lot of time and team effort (product owner, project managers, backend & frontend devs) to develop the contract in several iterations. After each iteration test the specification by mocking the API and clients before turning over to to implement your frontend app or cli client.
frontend specific endpoints along with General ones?
I would not do that, but I do not know you context. What does a frontend specific endpoint mean? If it means that as of today the endpoint should be only used by the frontend application but is of no use for the current cli client than I think it is just a matter of perspective. Make it a general endpoint and just use it by the frontend app. If it somewhat provides sensitive information that should be access only by the frontend you need to think about authentication and authorization. I recommend implementing Oauth2 for that.
create special endpoints for my frontend application for optimization, for example an endpoint to some statistics screenfrontend specific endpoints along with General ones?
I would suggest to implement all endpoints in your API and use OAuth2 as authentication. Use the scopes of the OAuth approach to manage authorization and access to different endpoints for each client (frontend app, cli).
You wrote you need to:
Change some of the basic API results structures that are not backward compatible and can break the Clients usage.
Try to avoid making breaking changes to your API. If it is used internally only you may be in control of the different clients accessing the API but even than the risk of breaking a client is high.
If you need to change existing behaviour you should think about API versioning or API evolution, which is a controversly discussed topic with a lot of different opinions and practices.
What is the best practice to design an API to meet these needs?
Design your resource representations so that they are forward and backwards compatible by design. Fundamentally, they are messages, so treat them that way; new optional fields with reasonable defaults can be added to the messages, but the semantics of a message element should never change.
If you dig through the old XML literature, you'll find references to ideas like Must Ignore and Must Forward -- those are the sorts of princples that also apply to the representations of long lived resources.
Create new resources when the existing resources cannot be conveniently extended to cover your new use case.
Can anyone explain difference between Swagger & HATEOAS. I can Search many time but no buddy can explain the proper detailed answer this two aspects.
The main difference between Swagger and HATEOAS IMO, which is not covered in the accepted answer, is, that Swagger is only needed for RPC'esque APIs. Such APIs, however, have actually hardly anything to do with REST.
There is a further, widespread misconception that anything exchanged via HTTP is automatically RESTful (~ in accordance with the REST archtitectural style), which it is not. REST just defines a set of constraints that are not choices or options but are mandatory. From start to finish. There is nothing wrong from being not RESTful, but it is wrong to term such an architecture REST.
Swagger describe the operations that can be performed on an endpoint and the payload (including headers and the expected representation formats) that needs to be sent to the service and also describe what a client might expect as response. This allows Swagger to be used both as documentation as well as testing-framework for the API. Due to the tight coupling of Swagger to the API it behaves much like a typical RPC service description, i.e. similar to WSDL files in SOAP or stub or skeletton classes in RMI or CORBA. If either the endpoint changes or something in the payload changes, clients implementing against a Swagger documentation will probably break over time just reintroducing the same problems typical RPC implementations have.
REST and HATEOAS, on the other side, are designed for disovery and further development. REST isn't a protocol but an architectural style to start with that describes the interaction flow between a client and server in a distributed system. It basically took the concepts which made the Web so successful and translated it onto the application layer. So the same concepts that apply to the browsable Web also apply to REST. Therefore it is no miracle that also HATEOAS (the usage of and support for links, link relations and link names) behave similar to the Web.
On designing a REST architecture it is benefitial to think of a state machine where a server provides all of the information a client needs to take further actions. Asbjørn Ulsberg held a great talk back in 2016 where he explains affordances and how a state machine might be implemented through HATEOAS. Besides common or standardized media-types and relation names no out-of-band knowledge is necessary to interact with the service further. In the case of the toaster example Asbjørn gave in his talk, a toaster may have the states off, on, heating and idle where turning a toaster on will lead to a state transition from off to on followed by a transition to heating till a certain temperature is reached where the state is transitioned to idle and switches between idle and heating till the toaster is turned off.
HATOAS will provide a client with the information on the current state and include links a client can invoke to transition to the next state, i.e. turning the toaster off again. It's important to stress here, that a client is provided by the server with every action the client might perform next. There is no need for a client implementor to consult any proprietary API documentation in order for a client to be able to interact with a REST service. Further, URIs do not have to be meaningful or designed to convey a semantical-expressive structure as clients will determine whether invoking that URI makes sense via the link-relation name. Such relation names are either specified by IANA, by a common approach such as Dublin Core or schema.org or by absolut URIs acting as extension attributes which might point to a human-readable description, which further might be propagated to the user via mouse-over tooltips or such.
I hope you can see by yourself that Swagger is only needed to describe RPC Web-APIs rather than applications that follow the REST architectural design. Messages exchanged via REST APIs should include all the information needed by a client to make informed choices on the next state transition. As such it is benefitial to design such message flows and interactions as state machine.
Update:
How are Swagger and HATEOAS mutually exclusive? The former documents your endpoints (making auto-generating code possible) and the latter adds meta-information to your endpoints which tell the consumer what they can do (i.e. which other endpoints are available). These are very different things.
I never stated that they are mutually exclusive, just that they serve two different purposes, where if you follow one approach the other gets more or less useless. Using both does not make any sense though.
Let's move the discussion to the Web domain as this is probably more easily understandable and REST is de facto just a generalization of the concepts used on the Web, so doing this step is just natural and also a good recommendation in terms of designing REST architectures in general. Think of a case where you as a user want to send some data to the server. You have never used the service before so you basically don't know how a request has to look like.
In Swagger you would call the endpoint documentation, select the option that most likely might solve your task, read up on how the request needs to look like and hack a test-case into your application that ends up generating a HTTP request that is sent to the respective location. Auto-generating code might spare you some hacking time, though you still need to integrate the stub classes into your application and test the whole thing at least once just to be safe. If you later on need to integrate a second service of that API or of yet an other API in general, you need to start from the beginning and look up the Swagger documentation, generate or hack the interaction code and integrate it into your domain. Plenty of manual steps involved and in cases of API changes you need to update your client as otherwise it might stop working.
In the Web example however, you just start your browser/Web client, invoke the respective URI that allows you to send the data to the server and the server will most likely send you a HTML form you just need to fill out and click the send button which automatically sends the request to the server which will start to process it. This is HATEOAS. You used the given controls to drive your workflow. The server taught your client every little detail it needed to make a valid request. It served your client with the target URI to send the request to, the HTTP method it should use and most often also implicitly the media type the payload should be in. In addition to that it also gave your clients a skeleton of the expected and/or supported elements the payload should contain. I.e. the form may require you to fill out a couple of input fields, select among a given set of choices or use some other controls such as a date or time picker value that is translated to a valid date or time representation for you. All you needed to do was to invoke the respective resource in your Web client. No auto-generation, no integration into your browser/application. Using other services (from the same or different providers) will, most likely, just work the same way so no need to change or update your HTTP client (browser) as long as the media-type request and responses are exchanged are supported.
In the case where you rely on Swagger RPC'esque documentation, that documentation is the truth on how to interact with the service. Mixing in some HATEOAS information doesn't provide you any benefits. In the Swagger case, carrying around additional meta-information that bloat up the request/response for no obvious reasons, as all the required information is given in the reference documentation, will, with some certainty, lead to people starting questioning the sanity of the developers of that service and ask for payload reduction. Just look here at SO for a while and you will find enough question asking on how to optimize the interaction further and further and reducing message size to a minimum as they process every little request and don't make use of response caching at all. In the HATEOAS case, pointing to an external reference is just useless as peers in such an architecture most likely already have support for the required necessities, such as URI, HTTP and the respective media types, implemented into it. In cases where custom media-types are used, support can be added at runtime via plug-ins or add-ons dynamically (if supported).
So, Swagger and HATEOAS are not mutually exclusive but the other gets more or less useless once you decided for one route or the other.
Swagger: Swagger aids in development across the entire API lifecycle, from design and documentation, to test and deployment. (Refer to swagger.io)
HATEOAS: Hypermedia as the Engine of Application State
An Ion Form is a Collection Object where the value member array contains Form Fields. Ion Forms ensure that resource transitions (links) that support data submissions can be discovered automatically (colloquially referred to as HATEOAS). (Refer to https://ionspec.org/)
One is a framework for supporting designing and testing for APIs, the other is an API design architecture.
Building a RESTful API is not a binary concept. That is why we use the Richardson maturity model in order to measure how RESTful an API is.
Based on this maturity model
At level 0 we provide mechanisms for client of the API to call some methods on the server (Simple RPC)
At level 1 we expose resources on the server so the client of the API can have direct access to the resources that it requires (exposing Resources)
At level 2 we provide a uniform way for the client of the api to interact with the API (exposed resources) and the HTTP protocol has these methods (using HTTP verbs to interact with resources).
the ultimate step is to make our api explorable by the client. HATEOAS provides such functionality (over HTTP) meaning that it adds relevant links and affordances (extra methods) that can executed on the resource so the client of the API can understand its behavior.
Based on these definitions in properly designed RESTful API there is no coupling between client and server and client can interact with the exposed endpoints an discover them.
On the other hand, swagger is a tool that helps you document your API along with some extra goodies (code generators).
I believe that Swagger (with the help of swagger Hub) provides services for implementing a RESTful endpoint with maturity levels up to 2. But it does not go any further and it does not provide proper support of HATEOAS.
You can define your resources and HTTP verbs in (json/yml) files. And based on this definition Swagger can generate API documentation and the extra goodies (client stubs and skeletal implementation of the server API).
For all those people who have worked with Java RMI, SOAP,... the extra goodies part is a reminder of old technologies where there was tight coupling between Client and Server because the stubs and skeletal implementations are all built based on the same API definition file.
I just successfully implemented a WF4 "versioning" system using WCF's Routing Service. I had a version1 workflow service to which I added a new Decision activity and saved it as a version2 service. So now I have 2 endpoints (with identical service contracts, i.e. all Receive activities are the same for both service) and a router that checks the content of a message (a "versionId" string on the object that all of my Receive's accept as an argument) to decide which endpoint to hit.
My question is, while this works fine when no changes are made to the service contract, how to I handle the need to add or remove methods from my service contract and create a version3 service? My original thought was, when I add the service reference to my client, I use the latest workflow service's endpoint to get the latest service contract. Then, in the config file, I change the endpoint I connect to to the router's endpoint. But this won't work if v1 and v2 have a different contract than v3. My proxy will have v3's methods and forget all about v1 and v2.
Any ideas of how to handle this? Should I create an actual service contract interface in my workflow solution (instead of just supplying a ServiceContractName in my Receive activities)?
If the WCF contract changes your client will need to be aware of the additional operations and when to call them. I have used the active bookmarks, it contains the WCF operation, from the persistence store in some applications to have the client app adapt to the workflow dynamically by checking the enabled bookmarks and enabling/disabling UI controls based on that. The client will still have to be updated when new operations are added to a new version of the workflow.
While WCF was young I heard a few voices arguing that endpoint versioning (for web services that is) should be accomplished by using a folder structure. I never got to the point of trying it out myself, but just analyzing the consequences of such a strategy seems to me as a splendid solution. I have no production experience of WCF, but am about to launch a rather comprehensive solution using version 4.0 of .NET (ASP.NET, WCF, WF...) and at this stage I would argue that using a folder structure to separate versions of endpoints would be a good solution.
The essence of such a strategy would be to never change or remove the contract of an endpoint (a specific version) until you are 100% sure that it is not used any more. While your services evolves you would just add new contracts and endpoints. This could lead to code duplication if one is not such a structured developer as one should be. But by introducing a service facade the duplication would be insignificant
I have been through the same situation. You can maintain the version by the help of custom implementation. save the Workflow Service URL in Database. And invoke them as per desire.
You can get the information about calling the WF Service with the URL by the client.
http://rajeevkumarsrivastava.blogspot.in/2014/08/consume-workflow-service-45-from-client.html
Hope this helps