We would like to extend Wiremocks standard behavior with our own extension that allows for state to be managed across multiple request/response pairs. In particular, we need to aggregate data from a number of requests that is then returned upon invocation of a particular URL. Our question is whether this is even possible with Wiremock and how it could be achieved.
The only idea we have had so far is to introduce custom handlebars that read from/write to global state (e.g., static fields), but is there some other extension mechanism that we are missing? We are aware that this is something you would typically implement with a standard application server, however we have already heavily invested (architecturally speaking) in Wiremock and would prefer to find a compatible solution.
I should add that we are actually using Wiremock Studio rather than plain Wiremock, which may affect what kind of extensions are possible.
Related
We have one POST API live in production. Now we have a requirement to accept Localization information and proceed with execution accordingly.
e.g. if distanceUnit is "KM" then process all incoming data in Kilometers.
There are three options I could think of to accept localization information.
As a http header i.e. localization: {"distanceUnit": "km"}
As a part of payload itself.
Request parameter.
I like the 1st option as
it doesn't change api contract.
It's easier for other apis to send this info in case they need to be localized in future.
Localization is a part of content negotiation so I don't think it should be part of payload/query parameter.
Any opinions here would be helpful to zero in on 1st or second option.
Thanks.
While accept-language, as indicated by the proposed link Kit posted, may be attempting, this only supports registered languages, maintained by IANA, the standadization gremium of the Web, but not certain generic configuration options out of the box. It may be attempting to default to miles for i.e. Accept-Language: us and use km elsewhere, American scientists may have certain issues with your application then if they want to use km instead of miles. But if this might not be the case, this clearly could be an option you might consider. In regards to custom HTTP headers, I wouldn't recommend using those as the problem with custom HTTP headers in general is that arbitrary generic HTTP clients do not support these which somehow contradicts the idea why one should use a REST architecture.
Let us transfer your problem to the Web domain for a second and see how we usually solve that task there. As REST is basically just a generalized approach to the common way we humans interact with the Web, any concepts used on the Web also apply to a REST architecture. Thus, designing the whole interaction flow as if your application interacts on a typical Web page is just common practice (or at least should be).
On the Web a so called Web form is used to "teach" a Web client (a.k.a. Browser) what data the server expects as input. It not only teaches the client about the respective properties the server either expects or supports for a certain resource but also which HTTP method to use, about the target URI to send the request to and about the media-type to use, which implicitly is often just given as application/x-www-form-urlencoded but may also be multipart/form-data.
The usage of forms and links fall into the HATEOAS constraint where these concpets allow clients to progress through their task, i.e. of buying an item in a Web shop or administrating users in a system, without the need of ever having to consult an external documentation at all. Applications here basically just use the build-in hypermedia capabilities to progress through their tasks. Clients usually follow some kind of predefined processes where the server instructs clients on what they need to do in order to add an item to the shopping cart or on how to add or edit a user while still just operating on a generic HTML document that by itself isn't tailored to the respective task at hands. This approach allows Web clients to basically render all kinds of pages and users to interact with those generic pages. If something in that page representation changes your browser will automatically adept and render the new version on the next request. Hence, the system is able to evolve over time and adapt to changes easily. This is probably one of the core reasons why anyone wants to use a REST architecture basically.
So, back to the topic. On the Web a server would advertise to a client that it supports various localization information with above mentioned forms. A user might be presented a choice or dropdown option where s/he can select the appropriate option. The user usually does not care how this input is transferred to the server or about the internals of the server at all. All s/he cares for is that the data will be available after the request was submitted (in case of adding or updating a resource). This also holds true for application in a REST architecture.
You might see a pattern here. REST and the browsable Web are basically the same thing. The latter though focuses on human interaction while the primer one should allow applications to "surf the Web" and follow allong processes outlined by the server (semi-)automatically. As such it should be clear by now that the same concepts that apply to the browsable Web also apply to REST and applications in that REST architecture.
I like the 1st option as ... it doesn't change api contract
Clients shouldn't bind to a particular API as this creates coupling, which REST tries to avoid at all costs. Instead of directly binding to an API, the Web and as such also REST should use contracts build on hyper media types that define the admissible syntax and semantics of messages exchanged. By abstracting the contract away from the API itself to the media-type a client can support various contracts simultaneously. The generalization of the media-type furthermore allows to i.e. express various different things with the same media type and thus increase the likelihood for reusage and thus a better integration support into application layers.
Supporting various media-types is similar to speaking different languages. By being able to speak various languages you just increase the likelihood that you will be able to communicate with other people (services) out of the box without the need of learning those languages before. A client can tell a server via the Accept header which media-types it is able to "speak( (a.k.a. process) and the server will either respond with either of these or respond with a 406 Not Acceptable. That error response is, as Jim Webber put it, coordination data that at all times tells you whether everything went well or in case of failures gives you feedback on what went wrong.
In order to stay future-proof I therefore would suggest to design the configuration around hypertext enabled media types that support forms, i.e. HTML forms, applicaiton/hal-forms+json or application/ion+json. If in future you need to add further configuration options adding these is just a trivial task. Whether that configuration is exposed as own resource which you just link to, as embedded part within the resource or not return to the client at all is also a choice you have. If the same configuration may be used by multiple resources it would be benefitial to expose it as own resource and then just create a reference from the resource to that configuration but as mentioned these are design decisions you have to make.
If the POST request body is the only place where this is used, and you never have to do GET requests and automatically apply any conversion, my preference would probably go to adding it to the body.
It's nice to have a full document that contains all the information to describe itself, without requiring external out-of-band data to fully interpret its meaning.
You might like to define your schema to always include the unit in relevant parts of the document, for example:
distance: [5, 'km']
or, as you said, do it once at the top of the doc.
We are building set of new REST APIs.
Let's say we have a resource /users with the following fields:
{
id: 1
email: "test#user.com"
}
Clients implement this API and can then update this resource by sending a new resource representation to PUT /users/1.
Now let's say we add a new property name to the model like so:
{
id: 1
email: "test#user.com"
name: "test user"
}
If the models the existing clients are using are to call our API not updated, then calls to PUT /users/1 will remove the new name property since PUT is supposed to replace the resource. I know that the clients could work straight with the raw json to ensure they always receive any new properties that are added in the API, but that is a lot of extra work, and under normal circumstances clients are going to create their own model representations of the API resources on their side. This means that any time any new property is added, all clients need to update the code/models on their side to make sure they aren't accidentally removing properties. This creates unneeded coupling between systems.
As a way to solve this problem, we are considering not implementing PUT operations at all and switching updates to PATCH where properties that aren't passed in are simply not changed. That seems technically correct, but might not be in the spirit of REST. I am also slightly concerned about client support for the PATCH verb.
How are others solving this problem? Was is the best practice here?
You are in a situation where you need some form of API versioning. The most appropriate way is probably using a new media-type every time you make a change.
This way you can support older versions and a PUT would be perfectly legal.
If you don't want this and just stick to PATCH, PATCH is supported everywhere except if you use ancient browsers. Not something to worry about.
Switching from PUT to PATCH will not fix your problem, IMO. The root cause, IMO, is that clients already consider the data being returned for a representation to follow a certain type. According to Fielding
A REST API should never have “typed” resources that are significant to the client.
(Source)
Instead of using typed resources clients should use content-type negotiation to exchange data. Here, media-type formats that are generic enough to gain widespread adoption are for sure beneficial, certain domains may however require a more specific representation format.
Think of a car-vendor Web page where you can retrieve the data from your preferred car. You, as a human, can easily identify that the data depicts a typical car. However, the media-type you most likely received the data in (HTML) does not state by its syntax or the semantics of its elements that the data describes a car, unless some semantic annotation attributes or elements are present, though you might be able to update the data or use the data elsewhere.
This is possible as HTML ships with a rich specification of its elements and attributes, such as Web forms that not only describe the supported or expected input parameters but also the URI where to send the data to, the representation format to use upon sending (implicitly given by application/x-www-form-urlencoded; may be overwritten by the enctype attribute though) or the HTTP method to use, which is fixed to either GET or POST in HTML. Through this, a server is able to teach a client on how a request needs to be built. As a consequence the client does not need to know anything else besides having to understand the HTTP, URI and HTML specifications.
As Web pages are usually filled with all kinds of unrelated stuff, such as adds, styling information or scripts, and the XML(-like) syntax, which is not every ones favourite, as it may increase the size of the actual payload slightly, most so-called "REST" APIs do want to exchange JSON-based documents. While plain JSON is not an ideal representation format, as it does not ship with link-support at all, it is though very popular. Certain additions such as JSON Hyper-Schema (application/schema+json hyper-schema) or JSON Hypertext Application-Language (HAL) (application/hal+json) add support for links and link-relation. These can be used to render data received from the server as-is. However, if you want a response to automatically drive your application state (i.e. to dynamically draw the GUI with the processed data) a more specific representation format is needed, that can be parsed by your client and act accordingly as it understands what the server wants it to do with it (= affordance). If you like to instruct a client on how to build a request support for other media-types such as hal-forms or ion need to be supported. Certain media-types furthermore allow you to use a concept called profiles, that allow you to annotate a resource with a semantic type. HAL JSON i.e. does support something like that where the Content-Type header may now contain a value such as application/hal+json;profile=http://schema.org/Car that hints the media-type processor that the payload follows the definition of the given profile and may thus apply further validity checks.
As the representation format should be generic enough to gain widespread usage, and URIs itself shouldn't hint a client as well what kind of data to expect, an other mechanism needs to be used. Link relation names are basically an annotation for URIs that tell a client about the purpose of a certain link. A pageable collection might return links annotated with first, prev, next and last which are pretty obvious what they do. Other links might be hinted with prefetch, that hint a client that a resource can be loaded right after loading the current resource finished as it is very likely that the client will retrieve this resource next. Such media-types, however, should be either standardized (defined in a proposal or RFC and registerd with IANA) or follow the schema proposed by Web linking, (i.e. as used by Dublin Core). A client that just uses the URI for an invoked link-relation name will still work in case the server changes its URI scheme instead of attempting to parse some parameters from the URI itself.
In regards to de/coupling in a distributed system a certain amount of coupling has to exist otherwise parties wont be able to communicate at all. Though the point here is, the coupling should be based on well-defined and standardized formats that plenty of clients may support instead of exchanging specific representation formats only a very limited number of clients support (in worst case only the own client). Instead of directly coupling to the API and using an undefined JSON-based syntax (maybe with external documentation of the semantics of the respective fields) the coupling should now occur on the media-types parties can use to exchange the format. Here, not the question of which media-type to support should be asked but how many you want to support. The more media-types your client or server supports, the more likely it is to interact with other peers in the distributed system. On the grand-scheme of things, you want a server to be able to server a plethora of clients while a single client should be able to interact with (in best case) every server without the need for constant adoptions.
So, if you really want to decouple clients from servers, you should take a closer look at how the Web actually works and try to mimic its interaction model onto your application layer. As "Uncle Bob" Robert C. Martin mentioned
An architecture is about intent! (Source)
and the intention behind the REST architecture is the decoupling of clients from servers/services. As such, supporting multiple media-types (or defining your own-one that is generic enough to reach widespread adoption), looking up URIs just via their accompanying link-relation names and relying on content-type negotiation as well as relying only on the provided data may help you to achieve the degree of decoupling you are looking for.
All nice and well in theory, but so far every rest api I encountered in my career had predefined contracts that changed over time.
The problem here is, that almost all of those so called "REST APIs" are RPC services at its heart which should not be termed "REST" to start with - this is though a community issue. Usually such APIs ship with external documentation (i.e. Swagger) that just re-introduce the same problems classical RPC solutions, such as CORBA, RMI or SOAP, suffer from. The documentation may be seen as IDL in that process without the strict need for skeleton classes, though most "frameworks" use some kind of typed data classes that will either ignore the recently introduced field (in best case) or totally blow up on invocation.
One of the problems REST suffers from is, that most people haven't read Fieldings thesis and therefore don't see the big picture REST tries to establish but claim to know what REST is and therefore mix up things and call their services RESTful which lead to a situation where REST != REST. The ones pointing out what a REST architecture is and how one might achieve it are called out as dreamers and unworldly when the ones proclaiming the wrong term (RPC over HTTP = REST) continue to do so adding to the confusion of especially the ones just learning the whole matter.
I admit that developing a true REST architecture is really, really hard as it is just too easy to introduce some form of coupling. Hence, a very careful design needs to be done that needs time and also costs money. Money plenty of companies can't or don't want to spend, especially in a domain where new technologies evolve on a regular basis and the ones responsible for developing such solutions often leave the company before the whole process had finished.
Just saying it shouldn’t be ‘typed’ is not really a viable solution
Well, how often did you need to change your browser as it couldn't interact with a Web page? I don't talk about CSS-stuff or browser-specific CSS or JS stuff. How often needed the Web to change in the last 2-3 decades? Similar to the Web, the REST architecture is intended for long-lasting applications for years to come, that supports natural evolution by design. For simple frontend-2-backend systems it is for sure overkill. It starts to shine especially in cases where there are multiple peers not under your control you can interact with.
When should I choose one over the other?
The way I see it API wrappers are so much simpler to use but I feel like there's something I'm not seeing, so can you enlighten me?
REST is a software design to decouple clients from APIs though it is often misunderstood as simple URI design thingy due to the fact that it is often based on HTTP.
Advantages of using APIs and clients that support REST is clearly, that clients are not coupled to any API in particular and are therefore tolerant to changes done on the server side like moving resources to different endpoints. Like a browser is able to present content of a sheer infinite number of web pages, true RESTful clients should behave identical and be able to communicate with any API that supports REST. It may learn on the fly how to deal with new content type by looking up some processing routines dynamically (similar to plugins of certain applications) or fallback to a default handling (reporting errors or presenting unknown data as plain text).
An API wrapper is often used to create clients that are limited to a certain API only. This simplifies development as the client can contain certain logic needed for interaction with the API like en/de-coding messages sent to or received from the service, list all available operations and similar stuff. Often URI endpoints are also either injected through properties or hardcoded into the application. Also, the content type is often limited to XML or JSON and the rules on how to treat responses are hardcoded into the client directly. All these steps however tightly couple the client to the API. In case the API changes (or is enriched by further endpoints) the API wrapper has to be updated and shipped to each consumer otherwise users either won't be able to use the API or make use of the latest features.
API wrapper are often tailor made for the usage of the API and are also often much simpler to implement. They, however, also require constant updating in cases the API itself is changing as the wrapper is incapable of handling these changes itself. REST clients on the other hand are far more complicate to develop as the client somehow has to know (or learn) the semantical meaning of a certain response and has to infer somehow how to act upon received responses. Some parts of it are yet an active field of research (at least in automated processing).
As you asked on when to use which: In cases where you have to create an all-purpose client, a REST client is for sure the right thing to do. However, identifying the correct semantical treatment will be the true chanllenge for these clients IMO. In cases where you only need to provide a client frontend for customers (or users) and not much change is expected on the API itself, a wrapper may be much easier to implement. However, please don't call such a client RESTful!
If I develop Booking REST service in Golang (i.e., in package booking). Is it a "GO way" to create BookingClient interface (backed up by struct) with business operations allowed, so that clients of my restful service would use BookingClient (imported from package booking) instead of sending http requests directly?
In general, no – if you provide a client in a particular language it'd only be a convenience, so (some) users can use your API easier. This of course assumes your client is well designed. I wouldn't provide merely an interface in Go just to indicate a set of possible API calls. This would be beneficial to a very narrow range of audience, probably for people developing a client for your API themselves, in programming language which just happened to be the same as implementation of your server. And even then they might not really like the idea of using the interface (e.g. they might only need a specific set of methods).
If you want to provide a client for your API, go ahead, do it, but separate it from the actual server (different package, maybe even different repo). In general one develops APIs over HTTP to allow for wide range of clients to access it, which could be written in any language. Instead of providing some interfaces I would invest my time in writing a good documentation.
In my opinion the answer to your question, assuming there is no more context provided, should be no different if you asked yourself if you should provide a client in, say, Python. The whole situation might change though if, for example, your API is used internally by your company and you develop mainly in Go.
It's usually preferable to do this, and most companies do, but provide documentation for working directly with the API. The main use case for that is people working with different languages than the ones you intended.
You can have a look at a new RESTful framework I wrote, that includes infrastructure to automatically compile clients with Go templates, although I haven't gotten to writing a Go client compiler. If you want to write one it would be greatly appreciated :) https://github.com/EverythingMe/vertex
Testing is important in Go, so writing testable code is something you should do. If you use direct http requests you will have a harder time writing unit tests, compared to using a mocked struct.
Is there any reason to use a Client rather than calling the functions that call the REST endpoints? It's usually harder to mock a bigger thing, such as a Client struct, rather than a group of small functions.
You should put the client at booking.Client to avoid repeating yourself (booking.BookingClient) and maybe rename Client to something more descriptive.
I get the benefits of changing link uris, but that's really not what this question is about.
What I mean by evolvability is adding new features to a service or modifying (when possible) existing ones and that's actually it.
SOAP isn't that bad as REST community tends to talk about it when it comes to evolvability. For example:
In REST we can add new rel - in SOAP we can add new method. Both
types of old clients will continue working with new services.
In REST we can add new form field and set its default value - in
SOAP we could have service arguments as some ServiceArgs class and
add a new field to ServiceArgs. That's ugly, but it works.
What are the evolvability examples when SOAP clients break and you can do nothing about it, while REST clients are handling the situation gracefully?
Thanks!
SOAP is a contract-based technology. The entire client/server interaction is written up and codified in a big document (the WSDL) and must be agreed upon and honored by both sides in order for things to work. If either side decides to add features, the other side must "evolve" in lock-step with it. Both sides are completely coupled, joined at the hip, glued together, married, for ever.
The typical approach to enhancing your SOAP services is to create new WSDL documents for the new versions of the service, while also maintaining the older ones. Another technique is to create a new interface to contain new methods and inherit from the old one. The approach you describe in #1 is IMO breaking the SOAP rules, because the client and server will now be using different contracts and it only works because additive changes (like new methods) can be shoe-horned in and most of the time things will work. The moment someone makes a destructive change then the client's contract will not match the server's and it's game over. It's a difficult process to manage, which is why most organizations opt to create entirely new WSDL for each new version of the API.
REST doesn't magically make all of these problems go away, but it makes things easier to manage by not forcing you to bundle your entire distributed system's "contract" into one artifact. You're using HTTP? Great, then you get to use all of the wonderful HTTP features that the web uses too: proxy servers, URLs, content negotiation, authentication, etc. You want to communicate using JSON encoding as well as XML? Knock yourself out. It's trivial to do in REST at any time, without affecting existing clients. You want security? Fine, start challenging for authenticated credentials using HTTP's in-built support for exactly that. All of these things (HTTP, JSON, etc) are standardized and described in different places and that's exactly how it should be.
SOAP combines the transmission protocol, location information, payload description, encoding choice and RPC methods into one ginormous document. If you want to make any change to anything in that list, you need a new document. Worse still, some of those things can't be changed at all.
REST separates those things out so that the pieces can evolve independently. Your URLs (or "URIs", to be more precise) are returned at runtime and assuming the client doesn't start to hardcode them are evolvable without any changes needed to the client. Additive changes to your media types are trivial if your documentation makes it clear that new fields may appear in the future. You've also got the option of versioning your media types, allowing the co-existence of v1/v2/v3... media types within your system, and the client can pick (using the Accept and Content-Type headers in HTTP) which one they want to use.
Ever heard the joke about the Porsche owner who buys a brand new car whenever the ashtray gets full? That's SOAP. What should be a trivial change requires a major overhaul. REST gives you the vacuum cleaner. You don't have to use it, but it sure is cheaper.